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AUTHORS’  NOTE 
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on  Distance  Learning  (DL).  An  ASB  study  panel  consisting  of  16  members  convened 
from  October  1996  through  the  completing  of  this  report  to  review  the  topics  of  the 
Army’s  Distance  Learning  Plan  (ADLP),  and  innovative,  leading  edge  developments  in 
Industry  and  Academe  Distance  Learning  initiatives.  The  Distance  Learning  panel 
focused  on  reviewing  the  Army’s’  Distance  Learning  Plan;  recommended  improvements 
to  the  plan;  and  characterizing  training,  development  and  delivery  beyond  the  plan. 

We  extend  our  sincere  appreciation  to  all  the  Study  Panel  members:  Dr.  Sue 
Berryman,  Dr.  Sung  Lee,  Dr.  Philip  Dickinson,  Ms.  Susan  Lowenstam,  Dr.  Edward 
Gerry,  Dr.  Warren  Morrison,  Dr.  Richard  Hartman,  Dr.  Harry  O’Neil,  Dr.  Walter 
Laberge,  Dr.  Peter  Lee,  Dr.  Karen  Pettigrew,  Dr.  Wilson  Talley,  Dr.  Harry  Tredennick 
and  LTG  (USA,  Ret.)  Ronald  Watts,  for  all  their  time,  effort  and  personal  sacrifice  during 
the  course  of  the  study.  The  briefing  of  the  study  result  was  well  received  by  the 
sponsors,  LTGs  Miller  and  Holder,  who  expressed  their  gratitude  for  the  Panel's  effort. 

We  would  also  extend  a  thank  you  to  COL  Francis  Coppola  of  the  USA 
Combined  Arms  Center,  our  staff  assistant  and  Reserve  Component  integrator  on  the 
study  subject.  His  support  and  dedication  to  educating  the  Distance  Learning  Panel  in  the 
critical  issues  surrounding  the  study  topic  impact  on  the  Reserve  were  invaluable. 


Dr.  Joseph  V.  Braddock 
Study  Chair 

LTG  (USA,  Ret.)  W.  Dutch  Shoffner 
Study  Co-Chair 
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Executive  Summary 


Business,  industry  and  academia  are  ahead  of  the  Army  in  use  of  DL  in  its  various 
modes.  Most  importantly,  they  are  increasingly  focusing  on  a  particularly  effective  DL 
mode  which  the  ASB  panel  terms  “DL-Collaborative”.  This  mode  permits  maximum 
flexibility  with  students  working  over  electronic  media  (most  likely  the  Internet),  seeking 
out  subject  matter  experts  and  working  with  each  other.  This  mode  also  allows 
instruction  or  learning  to  occur  at  a  time  and  place  convenient  to  the  student  and  has  been 
shown  to  enable  both  faster  and  better  learning. 

The  Army  has  already  established  the  world  class  training  methodology  in  its  CTCs, 
which  employ  a  synchronous  format.  The  Army  should  exploit  this  strength  by  utilizing 
the  Army’s  CTC  framework  to  establish  courses  delivered  via  the  asynchronous  DL- 
Collaborative  method. 

An  immediate  implication  for  the  Army  is  that  adoption  of  this  mode  would  have  a 
pronounced,  beneficial  effect  on  the  TTHS  account  with  realization  of  cost  transfer  in 
dollars  and  savings  in  available  man-years.  Training  could  be  provided  locally  at  a  time 
convenient  to  the  Soldier  as  well  as  his  commander  with  TDY  and  per  diem  costs  greatly 
reduced. 

Unlike  our  sister  services,  the  Army  has  developed  a  Distance  Learning  plan.  The  plan, 
however,  is  too  focused  on  an  older  learning  mode.  Known  as  “synchronous  remote 
distance  learning”,  this  mode  intellectually  perpetuates  the  traditional  teaching  paradigm 
of  an  instructor  presenting  material  to  an  assemblage  of  students  at  a  prescribed  time. 
This  DL  mode  also  carries  heavy  infrastructure  costs,  funding  the  ASB  panel  advises  be 
redirected  to  accelerating  courseware  development. 

For  these  reasons  the  panel  recommends  updating  the  current  Army  Distance  Learning 
Plan  to  include  extensive  use  of  the  DL-Collaborative  mode  and  accelerating  the 
implementation  of  Distance  Learning  to  expedite  realization  of  cost  transfer  and 
manpower  savings  and  reduce  the  overall  costs  of  implementing  DL. 

The  study  also  makes  comments  and  recommendations  regarding  management  issues, 
inclusion  of  the  civilian  workforce,  and  teaming  with  universities  and  industry. 
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The  Army  Science  Board  on  Distance  Learning 
Study  Panel 

This  Panel  emanated  from  an  inquiry  by  the  Assistant  Secretary  of  the  Army,  Financial 
Management  and  Comptroller  (ASA  FMC)  to  the  Army  Science  Board  to  include  a  study 
of  the  Army’s  Distance  Learning  Plan  as  one  of  the  1997  Summer  Study  topics.  The 
selection  of  the  Army  Science  Board  panel  for  the  Distance  Learning  (DL)  study  was 
completed  in  the  fall  of  1996  (Appendix  B  provides  a  complete  participant  list).  The  Co- 
Chairs  of  this  Study  Panel  effort  were  LTG  (Ret)  Wilson  Shoffner  and  DR  Joseph 
Braddock.  The  purpose  of  the  study  was  to  understand  the  Army’s  training  and 
educational  challenge  and  to  determine  potential  distance  learning  payoffs. 

It  was  imperative  for  all  involved  to  begin  with  a  common  reference  point  concerning 
distance  learning  basics.  Important  insights  were  gained  through  the  sampling  conducted 
by  the  panel.  These  insights  provided  a  clear  understanding  of  the  framework  present  in 
the  DL  environment  of  industry,  academe,  and  government,  and  as  a  result,  allowed  the 
Panel  to  review  and  compare  the  Army’s  DL  program  elements  and  initiatives. 

Through  this  phased  process,  the  Panel  deliberated  their  findings  and  conclusions 
concerning  the  current  and  potential  future  state  of  DL.  This  process  was  based  on 
analysis  of  a  suggested  vision,  developing  an  intellectual  framework,  and  specific 
recommendations  to  proceed. 
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TERMS  OF  REFERENCE  (TOR) 
(OUR  INTERPRETATION) 


•  Review  Army  Distance  Learning  Plan  (ADLP);  comment  on  priority  items  to  augment 
it 

•  Sample  industry,  academe,  &  government  DL  activities,  identify  promising  trends  & 
innovations,  recommend  improvements  to  ADLP 

•  Characterize  training,  development  &  delivery  beyond  ADLP  (201 0)  include  Force 
XXI  and  AAN 


Terms  of  Reference  (TOR) 

The  finalized  Terms  of  Reference  -  to  review  the  Army  Distance  Learning  Plan  (ADLP); 
Sample  Industry,  Academe  &  Government  DL  initiatives;  and  to  characterize  training, 
development  and  delivery  beyond  the  ADLP  were  developed  through  communication, 
cooperation  and  staffing  with  the  sponsors  and  other  interested  organizations  mentioned 
below: 


Organisation 
ASA  (FMC) 

ASA  (RDA) 

HQS TRADOC 

HQS  CAC 

HQS  DA  DCSOPS 

OCAR 

NGB 


Sponsor 
DR  Raynsford 
DR  Decker 
LTG  Miller 
LTG  Holder 
BG  Dubik 


Point  of  Contact(s) _ 

MS  Rebar 
LTC  Remias 
MR  Seger,  COL  Olson 
COL  Coppola 

COL  Gunlicks ,  LTC  Williams 
MR  Paxson 

MAJ  Bond,  MAJ  Gividen 


(Refer  to  Appendix  A  for  a  complete  copy  of  the  final  TOR.) 

The  Panel’s  endeavors  were  to  provide  insights  and  suggestions  relative  to  the  paradigm 
shift  from  Residential  Learning  (RL)  to  DL ,  from  predominantly  collective  RL  to 
individualized,  unit-based,  self  paced  DL,  except  where  soldierization  and  socialization 
demand  the  collective  residential  environment. 

The  Panel’s  assessment  is  that  Collaborative  and  Asynchronous  DL  is  the  highest  payoff 
DL  mode  to  accomplish  this  paradigm  shift  from  RL  to  DL. 
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THE  ARMY’S  TRAINING 
CHALLENGE  AND  OPPORTUNITY 

•  Army  remains  geographically  dispersed,  and  contingency 
deployments  are  high  (72,000  soldiers  in  42  countries) 

•  Units  have  substantial  turbulence,  much  due  to  meeting  institutional 
training  requirements 

•  Army  faces  a  major  challenge  in  meeting  training  needs:  the 
throughput  =  180,000  -  per  year  (Excluding  trainees  and  AIT) 

-  108,000  are  TDY-and-Return  (61 K  =  AC,  47K  =  RC) 

-  72,000  are  TTHS  (trainees,  transients,  holdees,  students) 

•  DL  offers  significant  opportunities  to: 

-  Reduce  unit  turbulence 

-  Reduce  direct  cost  (TTHS,  per  diem,  travel) 

-  Improve  unit  readiness  (MOSQ,  reclassification) 


V 

Distance  Learning 


Potential  Payoff 
$114M/yearand 
IQJjOO  MY 


The  Armv’s  Training  Challenge  &  Opportunity 

Training  is  an  enduring  function  in  the  Army.  Not  including  the  soldierization  of  new 
recruits,  about  50,000  officers  and  enlisted  are  in  training  at  any  given  time.  The  total 
annual  throughput,  excluding  Basic  Combat  Training  and  Advanced  Individual  Training, 
is  about  180,000.  The  Total  Army  (Active  and  Reserves)  strength  of  1,000,000  is  spread 
throughout  the  world;  most  are  in  the  Continental  United  States  (CONUS).  However, 
about  ten  percent  of  the  active  Army  is  outside  the  Continental  United  States  (OCONUS) 
at  sites  ranging  from  urban  areas  in  developed  countries  to  bivouacs  in  third  world 
countries.  All  of  these  soldiers  may  be  involved  in  individual  training  at  some  point  in 
time.  Today,  training  entails  moving  the  soldier  to  the  classroom.  This  results  in 
attendant  costs  (TDY,  TTHS,  etc.)  and  loss  of  readiness  equivalent  to  three  of  the  ten 
Army  combat  divisions.  In  comparison,  DL  offers  the  possibility  of  being  more  cost 
effective  as  well  as  enhancing  the  Army’s  readiness  by  reducing  unit  personnel 
turbulence. 

The  ADLP  gives  a  functional  definition  of  DL;  it  explains  what  DL  is  intended  to  do,  not 
what  it  is.  The  Panel  interpreted  this  functional  definition  as  a  way  to  instruct  the  soldier 
in  the  manner  least  disruptive  relative  to  unit  readiness.  DL  is  a  way  to  deliver  education 
using  the  most  appropriate  means  available:  harnessing  the  information  revolution, 
implementing  affordably  and  leveraging  all  financing  methods  -  buying,  leasing, 
establishing  partnerships  and  any  others. 
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Distance  Learning  is  not  a  panacea  for  the  Army.  For  example,  the  conversion  of  a 
civilian  into  a  Soldier  is  far  beyond  the  scope  of  any  present  DL  technique.  Therefore, 
soldierization  and  socialization  within  the  Army’s  environments  in  BCT  and  AIT  were  not 
considered  for  DL  in  this  study.  The  Panel  believes  that  DL  has  progressed  and  is 
advancing  so  rapidly,  that  harvesting  the  full  benefits  demands  a  cultural  change  within 
the  Army.  This  is  not  a  problem  peculiar  to  the  Army.  For  instance,  universities  and 
industries  have  taught  for  ages  under  the  old,  standard  paradigm  where  platform 
instructors  teach  students  by  the  method  of  listen  and  learn.  Most  people  are  a  product  of 
this  type  of  education,  so  this  paradigm  does  work.  Nevertheless,  there  are  two  factors 
that  call  upon  the  Army  to  change:  new  modes  allow  incredible  varieties  in  learning,  and 
the  Army  can  no  longer  afford  the  luxury  of  RL  in  terms  of  money  or  readiness.  Because 
the  implementation  of  DL  and  its  management  demands  a  cultural  change,  bold 
leadership  is  as  important  as  providing  the  necessary  resources. 

The  Panel  believes  DL  is  capable  of  providing  cost  transfer  of  $1 14M  &  savings  of  10K 
man  years  (MY). 
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Definitions  of  Teaching  Modes 

RL:  Residential  Learning;  DL:  Distance  Learning,  synchronous  and  asynchronous. 

RL-Schools  -  The  traditional  method  of  instruction;  bringing  the  students  to  the  teacher 
in  dedicated  facilities.  This  enhances  interaction  between  students  and  teacher. 
Disadvantages  of  this  mode  include  difficulty  of  scheduling;  loss  of  troops  from  unit;  and 
TDY  costs.  If  RL-Schools  were  somehow  eliminated  in  favor  of  DL  modes,  the  savings 
would  accrue  to  a  variety  of  accounts,  not  all  of  which  could  be  used  to  pay  DL  costs. 

DL-Remote  ('Classrooms)  -  Live  transmission  of  a  lesson  from  the  instructor’s  classroom 
to  satellite  classrooms.  This  can  involve  two  way  video  and  audio  for  maximum 
interaction  between  students  and  teacher.  Alternatives  range  down  to  one  way  video 
(high  bandwidth)  and  two  way  audio  (low  bandwidth)  or  similar  feedback  mechanisms. 
The  video  signals  can  be  transmitted  via  satellite  or  cable,  while  the  audio  can  go  over 
ordinary  telephone  lines.  Both  of  these  modes  are  synchronous,  occurring  in  real  time 
with  the  instructor  present. 

DL-Canned  ( material  anywhere)  -  The  material  of  the  lesson  is  fixed  (canned),  the 
earliest  DL  mode.  Examples  are  the  correspondence  course,  textbooks  and  self-paced 
texts.  Modem  transmission  modes  include  CD’s,  videotapes,  and  the  like.  This  is 
probably  the  most  inexpensive  mode  over  the  life  cycle  of  instruction. 
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DL-Collaborative  (Internet  or  the  like)  -  There  are  interactions  between  instructors  and 
students,  between  a  student  and  ‘intelligent  tutor’  (embedded  in  the  instructional 
medium),  or  among  students.  Presently,  this  is  the  most  expensive  mode,  but 
improvements  in  technology  are  likely  to  continue  to  drive  the  costs  down. 

Both  DL  -  Canned  and  DL-Collaborative  are  asynchronous,  occurring  anytime, 
anywhere,  with  the  instructors  piped  in  through  some  media,  normally  at  the  convenience 
of  the  student. 

The  Panel’s  view  of  distance  learning  includes  a  DL  intellectual  framework,  which  is  the 
basic  structure  for  supporting  decisions  about  learning.  Students  are  Soldiers  of  the  Total 
Army  and  represent  all  ranks,  both  enlisted  and  officer.  These  students  bring  highly 
variable  individual  attributes  to  the  learning  task:  prior  knowledge,  learning  skills,  and 
learning  motivation.  Learning  objectives  consist  of  what  the  Army  wants  the  students  to 
learn,  under  what  conditions,  and  to  what  standards.  Learning  objectives  are  a  major 
driver  of  all  other  decisions  about  DL. 

The  leaming/educational  framework  involves  decisions  about  elements  that  directly 
affect  the  learning  situation.  The  teacher/surrogate  includes  traditional  teachers  in  a 
classroom,  viewed  through  one-way  or  two-way  video  in  a  remote  classroom,  or  the 
individual  who  grades  and  comments  on  correspondence  coursework  or  responds  to 
Internet  queries.  It  includes  a  tutoring  function  that  is  embedded  in  computer-based 
courseware  (CW)  and  student  peers  who  are  linked  with  each  other  for  purposes  of  joint 
study.  CW  is  the  curriculum  or  subject  matter  with  which  the  student  directly  interacts. 

It  can  be  structured  in  a  variety  of  ways  and  conveyed  through  different  technologies  and 
media,  such  as  textbooks  and  other  printed  materials,  videotapes,  CD-ROMs,  and 
computer  programs. 

The  model  of  interaction  involves  two  decisions:  Are  there  interactions  between  the 
teacher  and  students,  among  students,  or  between  the  computer  program  and  the 
students?  Is  this  interaction  synchronous  or  asynchronous?  The  learning  environment 
constitutes  the  instructional  activities  assigned  to  students,  such  as  the  type  and  frequency 
of  homework  and  quizzes.  Evaluation  provides  feedback  on  students  and  courses. 

Student  evaluation  measures  skill  levels  at  the  start  of  the  course,  and  progress  during  and 
at  the  end  of  the  course.  Evaluation  of  courses  is  both  formative  (test/de-bug/fix)  and 
cumulative  (effects  on  students’  work  performance). 

The  cost-effectiveness  of  DL  is  driven  by  decisions  about  enablers:  The  selection  of 
technologies  (e.g.,  computers,  telecommunications),  facilities  (e.g.,  classrooms  versus  the 
kitchen  table),  and  type  of  media  (e.g.,  video,  computer-based  training,  intelligent 
tutoring  systems). 
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SCOPE  OF  SAMPLING 


V 

Distance  Learning 


DL  Users 

-  Educational  institutions  (1 00) 

-  Commercial  industry  (50) 

•  Manufacturing  and  Services 

-  Associations  (USDLA  AST&D  &  IEEE)  (3) 

-  Government  (20) 

•  Federal 

•  State 

•  Local 

DL  “Industrial  Base” 

-  Educational  institutions 

-  Infrastructure  &  related  services  (e.g.,  networks) 

-  Specialty  providers  (software,  courseware,  etc.) 

-  Technology  providers 


Scone  of  Sampling 

DL  is  used  by  many  organizations.  The  Panel  sampled  industry,  academe,  and 
government  through  site  visits,  telephone  conferences,  literature  searches,  and  surfing  the 
Web.  The  organizations  sampled  used  all  forms  of  DL;  some  have  done  so  for  25  years. 
Roughly  100  educational  institutions,  50  commercial  institutions,  3  associations  and  20 
government  institutions  were  asked  about  their  experiences  with  Distance  Learning. 

The  Panel  found  that  the  “Industrial  Base”  of  DL,  those  involved  in  providing  and 
improving  DL,  includes  some  of  the  users,  particularly  educational  institutions.  For 
example,  universities  moving  from  remote  classrooms  with  one  or  two-way  video/audio 
to  Internet  and  interactive  CD’s.  These  universities  purchase  or  develop  specialized 
software  and  hardware,  effectively  conducting  Research  &  Development  (R&D)  on  CW 
development  and  specialty  software.  After  product  testing  and  evaluation,  they  are  using 
their  own  DL  technologies  as  well  as  selling  them  to  industry  and  government. 

Other  elements  of  the  industrial  base  include  networks,  such  as  Regional  Bell  Operating 
Centers  (RBOC’s),  that  provide  satellite  channels  and/or  fiber  optic  cabling  for  a  fee. 

The  Panel  noted  that  many  commercial  software  applications  now  use  an  embedded 
interactive  tutor  to  supplement  or  replace  instruction  manuals.  While  such  tutors  are 
proprietary,  they  can  be  reverse-engineered  for  use  in  training  courses. 
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SCALE  OF  DISTANCE  LEARNING  IN 
THE  U.S.  _ 


21 .5M  ENROLLED 


UNIVERSITY  ARMY 


V 

Distance  Learning 


Scale  of  Distance  Learning  in  the  US 

Data  from  the  Bureau  of  Educational  Statistics  and  other  sources  indicate  that 
approximately  21.5  million  people  are  enrolled  in  DL  classes.  The  private  sector 
accounts  for  the  lion's  share  of  enrollees,  followed  by  the  educational  sector 
(Kindergarten  -  12th  grade),  and  state  and  local  governments.  Viewed  in  relationship,  the 
Army's  share  is  rather  small  and  will  remain  so.  The  educational  sector  is  expected  to 
grow  significantly  over  the  next  several  years.  This  in  part  means  that  the  Army  can 
expect  its  future  recruits  to  have  been  exposed  to  DL  in  one  form  or  another. 
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GENERAL  INSIGHTS  FROM  SAMPLING 


•  Education  and  industry  are  well  ahead  of  the  Army 

-  Use  RL  and  all  DL  modes 

-  Have  large  scale,  modern  and  efficient  programs 

-  Formulate  objectives  and  execute  programs 

on  basis  of  “Market  Forces”  and  affordability 

-  Recognize  courseware  (and  its  modernization)  as  a  major  challenge 

-  Exploit  technology  and  services 

•  Industry  outsources  for  needed  capabilities 

•  Education  employs  in-house  capabilities 


V 

Distance 


Both  employ  new  educational  paradigms 
emphasizing  asynchronous  collaborative  methods 


Learning 


General  Insights  from  Sampling 

As  previously  mentioned,  distance  learning,  in  some  form,  is  widely  used  in  the  industrial 
and  educational  sectors.  Choices  made  in  these  sectors  are  largely  driven  by  market  and 
budget  forces.  Outsourcing  is  the  first  choice  for  new  courses,  especially  in  industry. 
Furthermore,  educational  technology  has  guided  the  development  of  new  paradigms  that 
improve  the  efficiency  and  effectiveness  of  courseware.  In  other  words,  new  courses  are 
often  designed  based  upon  learning  principles  and  pre-registration  assessments  of  student 
needs.  Thus  there  is  an  intellectual  framework  for  learning  that  is  the  basis  for  the  design 
of  new  courseware.  There  is  also  an  “industrial  base”  that  produces  courseware  for  the 
mass  market.  Schools,  such  as  the  University  of  Kansas,  have  established  centers  that 
support  the  development  of  distance  learning  courses.  The  center  supplies  equipment, 
software,  and  expertise  to  professors  who  are  interested  in  offering  DL  courses. 
Courseware  creation  can  be  laborious  and  expensive,  despite  the  fact  a  number  of 
authoring  applications  have  been  developed  to  reduce  this  cost.  Both  sectors  employ  new 
educational  paradigms  emphasizing  asynchronous  collaborative  methods. 
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CURRENT  DISTRIBUTION 
OF  LEARNING  MODES 


UNIVERSITIES  INDUSTRY  ARMY 


V 

Distance  teaming 


Current  Distribution  of  Learning  Modes 

Distance  learning  is  growing  rapidly  in  both  universities  and  industry.  Currently, 
universities  are  still  teaching  largely  in  a  residential  mode.  However,  research,  especially 
in  the  areas  of  Computer  Science  and  Learning,  is  fueling  the  rapid  growth  of  DL. 
Universities  have  also  had  a  tight  coupling  with  the  Internet  and  have  been  recently 
exploiting  this  in  order  to  provide  DL.  Therefore,  the  Panel  expects  to  see  web-based  DL 
repositories.  There  are  a  growing  number  of  universities  and  consortia  offering  a 
substantial  number  of  on-line  DL  courses  and  degree  programs.  Examples  include  the 
California  Virtual  University,  California  State  University,  The  Western  Governors’ 
University,  and  The  University  on-line.  The  Stanford  Research  Institute  has  reported  the 
California  Virtual  University  will  offer  9,000  courses  by  1998. 

A  similar  growth  can  also  be  observed  in  industry,  although  the  delivery  modes  at  present 
show  different  proportions.  A  recent  study  reported  by  the  American  Society  for 
Training  and  Development  (ASTD)  provides  data  on  the  percentage  of  training  hours  by 
delivery  system.  The  study  shows  that  computer-based  training  budgets  are  on  the  rise, 
especially  for  off-the-shelf  CW.  The  market  for  multimedia  training  increased  by  53% 
from  1991  through  1995.  In  1996,  a  report  on  computer-based  training  found  use  of  the 
CD-ROM  as  a  delivery  vehicle  had  increased;  and  the  emergence  of  CD-ROM  and  LAN- 
based  delivery  systems  indicated  a  trend  towards  centralized  remote  DL.  The  use  of  the 
Internet  for  industrial  DL  has  grown  now  to  2%  of  the  overall  training  and  is  gaining  two 
new  users  per  minute. 
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The  active  Army  is  still  mostly  taught  by  RL  and  the  Army  Reserve  and  Army  National 
Guard  are  still  largely  on  paper-based  canned  instruction  (correspondence  studies).  The 
goal  of  the  ADLP  is  to  drastically  reduce  the  proportion  of  personnel  involved  in  both  RL 
and  canned  instruction,  and  to  execute  a  transition  to  remote  classrooms.  The  ADLP  also 
states  an  interest  in  DL-collaborative,  although  this  is  not  emphasized. 
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EDUCATION  AND  INDUSTRY 
DL  REMOTE  PROGRAMS 


•  Surrogate  teacher  is  real  and  remote 

•  Dominant  intellectual  framework  employed  is  RL 

•  Performance  (objectives) 

-  Able  to  meet  standards 

-  Able  to  do  some  ad  hoc,  just-in-time  training 

•  Efficiency 

-  Drawbacks  of  synchronous  accepted 

and  offset  by  competitive  advantages  gained 

-  Achieve  high  student-teacher  ratios  (50  to  100  to  1) 

•  Affordability 

-  Achieve  learning  standards  with  least  cost 

-  Enablers:  one  way  video,  data  compression,  and  low  bandwidth  reply 

-  Equip  with  low  end  cost  components 

-  Use  “Non-dedicated”  facilities 

Distance  Learning 


Education  &  Industry  DL  Remote  Programs 

Educational  institutions  and  industry  have  taken  advantage  of  telecommunications 
technologies  to  deliver  instruction  and  training  to  remote  sites.  Currently,  the  dominant 
foundation  of  these  remote  programs  is  framed  in  the  traditional  RL  mode,  except  the 
teacher  and  students  are  situated  at  different  locations. 

Relative  to  universities,  this  type  of  remote  program  facilitates  teaching  courses  to  a 
wider  audience,  including  non-traditional  students  such  as  mature  adults  who  are 
employed  full  time  (this  could  be  the  only  means  of  satisfying  this  group’s  educational 
needs).  Therefore,  a  higher  student  to  teacher  ratio  is  achieved. 

In  the  case  of  industry,  the  remote  training  program  provides  a  valuable,  competitive 
advantage  when  it  becomes  necessary  to  train  employees,  in  a  timely  manner,  in  order  to 
meet  minimum  skill  standards.  For  example,  when  a  new  product  is  introduced  or  a 
quick  adjustment  in  the  normal  business  routine  is  required,  a  large  number  of  employees 
can  be  educated  rapidly  on  an  ad  hoc  basis  by  adopting  the  “just-in-time”  (JIT)  training 
concept.  Some  businesses  have  adopted  this  method  of  remote  training  by  establishing  a 
network  of  low-cost  facilities  where  employees  can  be  trained  without  leaving  the 
workplace.  A  good  example  of  this  is  the  network  of  “classrooms”  established  in 
automobile  dealerships. 

Even  though  this  type  of  remote  program  is  synchronous  by  nature,  the  drawbacks  are 
offset  by  the  competitive  advantages  it  provides.  For  example,  the  start-up  costs  of  the 
program  are  comparatively  low  when  consideration  is  given  to  the  number  of  students  the 
program  can  reach.  The  enabling  technologies  —  one-way  video,  data  compression,  and 
low  bandwidth  reply  —  are  readily  available  and  widely  used  commercially. 
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INSIGHTS  FROM  EXAMINATION  OF  EDUCATION 
ND  INDUSTRY  LEADING  EDGE  DL  PROGRAMS 


STn 


•  All  are  dominantly  asynchronous  (some  have  small-to-modest  RL  fractions) 

•  Concentrated  in  sectors 


•  Examples 

-  Business  school  programs 

-  Engineering  and  information  science  programs 

-  Medical  sector  specialties  e.g.,  radiology 

-  Chip  design  and  manufacturing 

-  Some  software  products 

•  Characteristics 

-  On-demand,  rich  and  varied  interaction 

-  Intellectual  framework  (often  buried) 

-  Varied  teacher  surrogate 

-  Multiple  interaction  modes  via  Internet 

-  Motivators  and  tutors  are  sometimes  live  (on  demand),  but  mostly  embedded 

-  Affordable  personalized  technology  (PC,  Internet) 

-  Sophisticated  courseware  and  evaluation 

Distance  Learning  _ _ _ _ _ 


Insights  from  Examination  of  Education  and  Industry  Leading  Edge  DL  Programs 

The  leading  edge  DL  programs  in  education  and  industry  are  mostly  asynchronous  and, 
thus,  predominantly  DL-Canned  or  DL-Collaborative.  These  programs  are  concentrated 
in  a  few  relatively  technical  occupations,  such  as  business  school  or  medical  sector 
specialties.  These  programs  tend  to  use  affordable,  personalized  technology,  like 
personal  computers  and  the  Internet.  Generally,  they  use  sophisticated  computer-based 
CW  with  a  student  focused  intellectual  framework.  Leading  edge  DL  programs  are  on 
demand  -  available  when  the  students  need  them.  They  can  be  accessed  through 
multiple  interaction  modes,  such  as  teacher  with  student  and  student  with  student.  They 
employ  highly  developed  computer-based  tutors.  The  more  advanced  leading  edge  DL 
programs  tend  to  be  supported  with  digital  libraries,  and  evaluations  of  the  student’s 
progress  relative  to  learning  objectives. 
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ALTERNATE  TEACHING  TECHNIQUES  &  RESULTS: 
_ (Carnegie  Mellon  &  Indiana  University) _ 


High  School  &CoUege  Students 


M  (Ss££>  12 

k‘i  ^  '  - 
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Alternate  Teaching  Techniques  &  Results  (Carnegie  Mellon  &  Indiana  University) 

This  chart  reflects  the  advantages  provided  through  redesigning  and  re-engineering  the 
way  the  student  is  taught.  For  example,  an  existing  RL  course  is  examined  for  critical 
tasks  and  then  modified  to  reflect  these  objectives.  It  is  then  updated  to  reflect  current 
policies,  regulations  and  laws.  Through  performing  this  process,  the  universities  have 
achieved  between  25%-30%  reduction  in  course  length.  This  is  independent  of  the  type 
of  media  selected  for  the  final  output  -  updated  RL  courses,  Computer  Based  Instruction 
(CBI),  etc. 

The  Carnegie  Mellon  University  (CMU)  Human  Computer  Innovative  Interface  Lab, 
redesigns  courses  using  a  systematic  approach  to  training,  similar  to  the  Army’s  Systems 
Approach  to  Training  (SAT).  CMU  re-engineers  the  course  and  applies  techniques 
emphasizing  how  students  learn.  Enablers  such  as  the  Internet  or  Chat  Room,  are 
provided  for  students  to  collaborate  with  each  other.  Only  after  applying  this  process 
does  CMU  select  the  output  media  for  the  course.  In  this  case,  CBI  is  used  as  an  enabler. 
Thus,  the  result  is  an  additional  savings  in  time  and  an  increase  of  .5  standard  deviation  in 
student  performance.  CMU  has  also  designed  Intelligent  Tutors  for  complex  courses 
achieving  a  75%  reduction  from  the  original  course  length  and  an  increase  of  a  full  1.0 
standard  deviation  in  student  performance. 
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LEARNING  MODE  COMPARISONS 


J'N 


SYNCHRONOUS 


ASYNCHRONOUS 


RL 

-  Focus  on  individual 
1  Lecturer  provides  or  selects 

-  Interpretation 

-  Motivation 

-  Courseware 

(Books  &  Notes) 

-  Evaluation 


DL  Remote  1 

•  Individual 

•  Same  as  for  RL 


DL  Canned 

•  Individual 

•  Student  provides 

-  Interpretation 

-  Motivation 


Lecturer  Provides 
-  Courseware 
(Including 
Learning 


DL  Collaborative  1 

•  Individual  and  Group 

•  Student  Peers  and 

Facilitators  provide 

-  Interpretation 

-  Motivation 

•  Lecturer  approves 
Courseware  which 
includes  self-testing, 
tutoring  and  motivation 

•  Lecturer  provides 


V 


-  Evaluation 

Evaluation 

Institution  Provides 

-  Classroom 

Same  as  for  RL  + 

-  Faculty 

-  Faculty 

-  Faculty 

-  Overhead 

-  Video 

Network 

-  Degree 

-  Degree 

-  Degree 

-  Non-dedicated 
Classroom 

-  Internet” 

Distance  Learning 


Learning  Mode  Comparisons 

In  the  synchronous  RL  mode,  the  focus  is  on  the  individual.  The  lecturer  provides  or 
selects  for  the  student  the  interpretation,  motivations,  CW  -  books  and  notes  -  and 
evaluation.  The  institution  provides  the  classroom,  the  faculty,  all  overhead  and  awards 
the  degree. 

The  synchronous  DL  remote  mode  is  relatively  the  same  as  current  RL,  except  the 
student  is  exposed  to  a  video  network  through  some  non-dedicated  classrooms 
coordinated  by  the  institution. 

The  asynchronous  DL-Canned  mode  also  focuses  on  the  student,  however  it  is  the  student 
who  provides  the  interpretation  and  motivation.  The  lecturer  provides  the  CW,  including 
the  learning  aids  and  the  evaluation.  The  institution  only  provides  the  faculty  and  the 
award  of  the  degree. 

As  education  transitions  to  the  asynchronous  DL-Collaborative  mode,  a  slight 
modification  in  focus  to  both  the  student  and  to  the  group  occurs.  Students,  student 
peers,  and  facilitators  provide  interpretation  and  motivation.  The  lecturer  approves  the 
CW,  which  includes  self-testing,  tutoring,  motivation,  and  evaluation.  The  institution 
provides  the  faculty,  and  awards  the  degree,  but  the  Internet  or  Internet  like  media 
delivers  the  instruction. 
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PARALLELS  BETWEEN  CTC 
AND  COLLABORATIVE  DL 


cm 

•  Free  play  OPFOR 

•  Ground  truth 

•  Observer  controllers 

•  AARs 


V 


•  Team  training  and 


Distance  Learning 


Collaborative  DL 

Peer  competition  and  imbedded 
objective  self-testing 

Imbedded  Objective  Self  Evaluation 

Tutors  and  Motivators 
(Real  or  Technically  Based) 

Peers  and  SME  interactions  and 
use  of  results  of  imbedded 
objective  testing 

Peer  collaboration 


Parallels  Between  the  Army’s  Combat  Training  Centers  (CTCs) 
and  Collaborative  DL 


To  determine  an  appropriate  framework,  the  Panel  reviewed  leading  edge  programs 
within  the  Army,  which  might  serve  as  examples  to  provide  guidance  for  the  ADLP. 
Close  examination  reveals  interesting  parallels  between  the  methods  of  the  Army’s 
Combat  Training  Centers  (CTCs)  and  the  teaching/leaming  practices  of  universities  and 
industries.  As  an  example,  the  Army’s  Observer  Controllers  have  a  function  similar  to 
that  of  university’s  tutors.  This  chart  summarizes  the  pairings  of  similar  functions  in  the 
Army  with  those  in  academic  and  industrial  establishments.  The  Army  and  the  civilian 
sector  have  a  great  deal  to  leant  from  each  other  and  will  achieve  mutually  beneficial 
gain  through  collaboration  in  distance  learning. 

Although  there  are  many  similarities  between  the  operations  of  the  CTCs  and  the 
teaching/leaming  methods  of  universities  and  industries,  there  are  also  some  fundamental 
differences,  especially  when  the  functions  relate  to  DL.  The  CTCs,  by  nature,  operate  at 
fixed  sites  and  in  a  synchronous  mode  following  prescribed  schedules.  Universities  and 
industries,  on  the  other  hand,  have  developed  and  are  utilizing  a  delivery  mode  of 
teaching  to  distributed  sites  adopting  both  synchronous  and  asynchronous  modes 
(allowing  students  to  learn  on  their  own  schedule).  When  DL  is  judiciously  applied,  it 
can  lead  to  substantial  training,  readiness  enhancement  and  cost  savings. 

For  DL  the  intellectual  framework  represented  in  the  CTCs  is  more  leading  edge  for 
training  purposes  than  current  training  practices  in  industry  and/or  education.  However, 
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the  best  practices  of  industry  and/or  education,  through  asynchronous  applications,  are 
less  costly,  more  widely  distributed  and  should  be  combined  with  the  collaborative 
techniques  employed  at  the  CTCs.  Further,  a  strategy  must  be  developed  for  acquiring 
technologies  that  change  rapidly  and  which  also  deals  with  obsolescence  for  CW  and 
technology. 
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INSIGHTS  PERTINENT  TO  THE  ARMY 


•  Industry  and  education  experience  shows  conclusively  that  collaborative  learning 

produces  much  better  learning  in  a  much  shorter  time 

•  The  Army  is  clearly  at  the  forefront  of  synchronous  collaborative  learning 

with  the  techniques  it  employs  in  its  CTCs 

-  After  Action  Reviews 

-  OCs  as  Facilitators  and  Tutors  (Not  Lecturers) 

-  Individual  and  Group  Learning  and  Motivation 

•  Similarly,  the  Army  has  the  most  advanced  synchronous  collaborative  courseware 

analogs  in  its  preparatory  simulation  tools  (BCTP,  SIMNET,  UCOFT,  CAS3,  etc.) 

•  The  Army’s  DL  program  should  combine  both  its  collaborative  methodologies 

and  the  asynchronous  successes  of  industry  and  education 

Distance  Learning 


Insights  Pertinent  to  the  Army 

Industry  and  Education  experience  shows  conclusively  that  the  collaborative  learning 
method  produces  a  superior  learning  environment.  The  DL-collaborative  method 
provides  superior  education  within  a  considerably  shorter  time  frame.  The  Army  is  at  the 
forefront  of  applying  the  synchronous  collaborative  learning  paradigm  through  its  CTC 
techniques.  The  employment  of  After  Action  Reviews  (AARs)  and  Observer  Controllers 
(OCs)  create  individual  and  group  learning  and  motivation.  The  Observer  Controllers 
function  as  facilitators  and  tutors,  not  lecturers,  thereby  providing  orchestration  to  the 
learning  environment.  Similarly,  the  Army  has  the  most  advanced  synchronous 
collaborative  CW  analogs  in  its  preparatory  simulation  tools,  demonstrated  in  the  Battle 
Command  Training  Program,  Simulations  Network,  Unit  Conduct  of  Fire  Trainer,  and 
the  Combined  Arms  &  Services  Staff  School.  The  ADLP  should  combine  both  its 
collaborative  methodologies  and  the  asynchronous  successes  of  industry  and  education. 

The  DL  framework  is  composed  of  students;  learning  objectives;  teacher/surrogate;  CW; 
model  of  interaction;  learning  environment;  evaluation;  and  the  enablers  of  technology, 
facilities,  media,  and  so  forth.  The  enablers,  for  example  —  technology  —  should  be  the 
best  of  the  available  options  to  facilitate  subset  testing  for  cost-effectiveness. 
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If  you  train  as  you  fight  - 

And  you  train  at  the  CTC’s  in  integrated  combined  arms  teams 

Then  why  not  learn  and  educate  on  the  same  model? 

...and  map  the  CTC  model  into  the  institutional  framework  and 
leverage  emerging  DL  techniques  and  technology 

today  tomorrow 


Thesis 


The  education  programs  of  the  Army  are  not  isolated.  These  programs  are  required  for 
the  Army’s  development  of  doctrine,  taught  in  the  Army’s  RL  Schools,  and  trained  by 
units  in  the  field.  The  Army  has  a  great  deal  of  experience  in  delivering  high  quality 
training  to  integrated  combined  arms  teams  at  the  CTCs.  The  civilian  sector  has  some 
parallels  in  this  field,  but  the  Army  clearly  excels.  The  Army  should  take  advantage  of 
this  strength  and  combine  it  with  rapidly  developing  telecommunications  technologies  to 
structure  an  exceptional  DL  program.  The  Panel  recognizes  the  intersection  of 
Doctrine/School/Field  is  kept  small  because  of  the  turbulence  in  Army  assignments, 
which  has  no  parallel  in  the  civilian  sector.  In  time  of  war,  of  course,  the  three  areas  on 
the  Venn  diagram  merge,  and  the  area  of  intersection  grows  larger. 
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V 


ARMY  DISTANCE  LEARNING  PLAN 
(ADLP) 


Purpose 

-  Take  training  to  the  soldier 

-  Improve  readiness  of  the  total  force 

-  Framework  to  implement 

•  TASS  -  Classroom  XXI  -  Army  modernization  training 

•  Training  network  for  contingency  operations  &  soldier  growth 

-  Builds  on  successes  -  Bosnia,  Iowa  NG,  etc. 

ADLP  includes: 

-  Policies,  procedures,  and  responsibilities 

-  Common  telecommunications  environment 

-  Prioritized  training  materials  for  reconfiguration,  development,  and 
delivery 

-  Army  modernization  training 

-  Establishment  of  global  networks 

-  Provision  for  funding  strategy  and  resourcing  model  for  system 
hardware  and  school  workload 


Distance  Learning 


The  Army  Distance  Leaning  Plan  (ADLP) 

The  ADLP  proposes  to  deliver  education  to  the  soldier  at  the  right  place  and  right  time  by 
the  use  of  DL.  The  main  purpose  of  the  plan  is  to  improve  readiness  of  the  Total  Force 
and  it  enables  training  the  Total  Army  to  the  same  standard.  The  ADLP  will  support 
Army  Training  XXI  (Army  training  that  corresponds  directly  to  Force  XXI 
implementation)  and  be  included  in  the  Total  Army  School  System  (TASS)  by  using  DL 
technologies  and/or  techniques. 

The  ADLP  includes  policies,  procedures,  and  responsibilities  for  implementation.  The 
plan  details  a  specific  configuration  for  hardware  and  infrastructure  as  well  as  details  and 
schedules  for  course  selection,  course  conversion,  and  media  selection,  with  their  related 
costs  and  their  projected  savings. 
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PROGRAM  DERIVED  FROM  ADLP 


Facilities  (7  types) 

Components  Cost 

-  DL  centers  (16) 

-  DL  satellites  (7) 

-  DL  facility  (145) 

-  Mobile  DL  facility  (11) 

-  CTC  mobile  DL  facility  (3) 

-  OCONUS  fixed  site  DL  facility  (11) 

-  Deployable  DL  package  (4) 

-  Networks  $11 0M 

-  Courseware  $269M 

-  Personnel  $45M 

-  Hardware  $242M 

-  O&M  $174M 

Total  (‘98-’10)  $840M 

V 

Distance  Learning 


•  BL  intellectual  framework 

•  Synchronous  DL  remote  classrooms  and  their 
characteristics  in  an  Army  setting 

•  High  end  enablers  and  technology 

•  Very  lo  w  student-teacher  ratios 


The  Program  Derived  from  the  ADLP 

The  principal  portion  of  the  ADLP  transforms  some  RL  into  a  synchronous  DL  setting 
with  a  heavy  emphasis  on  remote  classrooms.  Traditional  classroom  teaching  is 
transferred  to  remote  sites  by  technologies  such  as  VTT  (two-way  video  and  audio  or 
data  transmission).  To  do  this,  wide  bandwidth  networks  and  associated  hardware  are 
needed.  Remote  classrooms  with  high-end  enablers  are  very  expensive.  The  plan  leans 
heavily  on  VTT,  robust  electronic  networks,  and  electronic  training  platforms.  While 
42%  of  the  total  cost  of  the  plan,  $840  million,  is  projected  for  networks  and  other 
hardware  over  the  13-year  funding  profile,  only  32%  is  allocated  for  course  conversion. 
The  remaining  funds  are  designated  for  personnel,  and  operations  and  maintenance 
(O&M).  The  current  draft  of  the  ADLP  is  based  on  a  traditional  RL  framework  that  is 
transformed  to  a  synchronous  DL  remote  classroom  mode. 

Although  not  specifically  written  in  the  plan,  in  concept,  the  Army’s  DL  program 
supports  the  continuance  of  the  classical  RL  framework  -  the  classical  paradigm. 
Students  may  not  be  physically  situated  in  the  same  room  as  the  instructor,  but  they  will 
observe  the  live  instructional  delivery.  The  students  will  see  the  image  of  their  instructor 
through  video  screens,  with  teacher-student  interaction  occurring  via  two-way  video 
teleconferencing  technology.  In  the  non-resident  TASS  DL  classroom  configurations, 
backup  instructors  and/or  assistant  instructors  will  also  be  present.  Evaluation  of  the 
student’s  progress  is  through  standard  classroom  evaluation  techniques,  such  as  paper- 
and-pencil  tests. 
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This  model  requires  expensive,  high-end  technology  (two-way  video)  and  physical 
classrooms  where  students  gather  at  remote  sites.  The  ADLP  staffing  model  proposes 
one  teacher  instructing  the  course  with  an  assistant  (monitor/  teacher)  for  every  15 
students  at  each  remote  site.  This  produces  a  very  low  teacher  to  student  ratio  (1:  7.5). 

Five  hundred  twenty-five  courses  are  projected  for  conversion  to  multimedia,  with  a  mix 
of  two-thirds  RL  and  one-third  DL  for  each  course.  The  course  model  does  not  project 
any  course  delivered  completely  via  DL.  The  525  courses  were  selected  through  a 
careful  process  from  about  2,500  courses.  The  process  consisted  of  consulting  with  the 
TRADOC  Proponent  Schools  and  AC/RC  commands  to  evaluate  each  course  relative  to 
factors  of:  readiness  enhancement;  MOSQ  density;  course  quotas/seats  and  delivery 
media.  Task  analysis  within  each  course  consists  of  an  effort  to  eliminate  redundancy, 
and  to  match  resident,  non-resident  and  correspondence  options  to  achieve  one  standard. 
Programs  of  instruction  are  then  prepared  and  Army  Training  Support  Packages  (TSP) 
are  completed.  This  initial  process  is  culminated  through  adaptation  into  the  Army’s 
Doctrine  and  Training  Digital  Library  (ADTDL).  After  completion  of  these  steps,  the 
course  will  be  developed  to  accommodate  appropriate  DL  technologies.  This  entire 
process  may  be  sequential  or  concurrent. 

The  funding  for  the  program,  especially  the  CW  conversion  component,  is  stretched  over 
13  years  because  of  a  shortage  in  funding  and  subject  matter  specialists.  The  result  is  that 
the  savings  associated  with  even  the  proposed  modest  shift  from  RL  to  DL,  which  the 
program  envisions,  cannot  be  reaped  for  many  years. 

The  Panel  views  the  ADLP  as  applying  primarily  a  RL  Intellectual  framework.  The 
Army  setting  promotes  synchronous  DL  remote  classrooms  with  their  inherent 
characteristics.  The  ADLP  emphasizes  high-end  enablers  and  technology  against  a  very 
low  student-teacher  ratio. 
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TODAY’S  ORGANIZATION  FOR  A  DL 
CAPABILITY  (SIMPLIFIED) 


-Tn 


TRADOC 

Program  Architect 


TPIO 


PEO  RCAS  PEO  STAMIS 

(NG  PGM)(Oct95)  (ADLP)(Apr97) 


TATS-C 


Classroom  XXI 


•  Facilities 

•  Equipment 

•  Networks  &  Access 


•  Facilities  (inch  non- 
TRADOC  classrooms) 

•  Equipment 

•  Networks 


v 

Distance  Learning 


•  Unify  and  update 
existing 
courseware 


• TRADOC 
classroom 


ADLP 


•  DL  education  methodology 
(How  students,  teachers, 
technology  and  courseware 
come  together) 

•  DL  courseware 

•  Instructors  and  monitors 


Today’s  Organization  for  a  DL  Capability  (simplified) 

The  Army  manages  the  DL  program  via  a  diffuse  management  process.  The  Army’s  CIO 
and  technical  architecture  are  managed  through  two  separate  Program  Executive  Offices. 
A  General  Officer  Steering  Committee  (GOSC)  provides  general  advice  to  the  Program 
Executive  Office  (PEO)  for  Standard  Army  Management  Information  Systems 
(STAMIS).  PEO  STAMIS  has  the  responsibility  for  the  infrastructure  portion  of  the 
ADLP.  Another  PEO  has  the  responsibility  for  the  Reserve  Component  Automation 
System  (RCAS),  a  National  Guard  and  Army  Reserve  system.  The  ADLP  program 
architecture  falls  under  the  Commanding  General  (CG)  Training  and  Doctrine  Command 
(TRADOC).  TRADOC  has  the  overall  control  of  the  funding  for  the  CW  conversion. 
Both  PEOs  acquire  facilities,  equipment,  and  network  access  for  the  respective  clients. 
TRADOC  is  responsible  for  three  DL  activities:  TATS-C  (Total  Army  Training  System- 
CW),  Classroom  XXI,  and  ADLP.  A  separate  Defense  Advanced  Research  Projects 
Agency  (DARPA)  program.  Simulations  in  Training  and  Readiness  (SIMITAR),  was 
funded  for  the  last  few  years  to  explore  simulation  capabilities  with  DL  applications.  It 
appears,  however,  that  this  program  will  cease  when  current  funding  expires  because  a 
major  Army  organization  has  not  expressed  willingness  to  provide  further  support. 
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Example  of  Reengineering.  Asynchronous  &  Collaborative 

This  chart  shows  the  examples  demonstrated  through  course  redesign  at  Fort  Eustis, 
Carnegie  Mellon  University  (CMU)  and  Indiana  University  side  by  side.  The  savings  are 
not  achieved  from  the  transition  of  the  CW  onto  a  Computer  Based  Instruction  (CBI) 
platform.  The  savings  are  achieved  by  performing  a  functional  process  analysis  of  the 
critical  learning  objectives  for  the  course,  applying  motivation  and  collaboration 
techniques,  and,  when  needed,  developing  Intelligent  Tutors  to  assist  the  student  in 
learning  the  material. 
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PAYOFF  POTENTIAL  FOR  DL 


STRUCTURE  REDUCTIONS  PER  DIEM  COST  AVOIDANCE 


Source  Data:  ATRRS  -  Army  Training  Resource  Requirement  System 

Assumptions: 

•  Savings  based  upon  steady-state  objective  for  525  RL  courses 

•  30%  reduction  of  RL  student  instructional  time  with  a  cost  avoidance  in  per  diem  @  $30/day 


V 


Per  Diem  Savings  Can  Offset  Cost  of  Implementing  DL 


Distance  Learning 


Pavoff  Potential  for  DL 

As  far  as  the  Total  Army  is  concerned,  course  conversion  to  DL  saves  dollars.  The  faster 
the  conversion,  the  more  money  saved.  Readiness  is  improved  as  well. 

The  current  ADLP  calls  for  the  conversion  of  525  RL  courses  from  2500  mixed  RL  and 
DL  courses.  The  TRADOC  cost  analysis  projects  a  total  cost  of  $839.7M  projected  over 
a  13  year  funding  profile.  The  annual  cost  ranges  from  $37.5M  to  $93.5M  with  an 
average  annual  cost  of  $64. 6M  over  the  life  of  the  program.  The  savings  projections 
involve  complex  formulas  to  determine  per  diem,  travel,  permanent  change  of  station 
(PCS),  lease  and  usage  fees  (Tele-training  Network  TNET  and  Video  Tele-conferencing 
VTC),  reduced  student  load  and  instructor  training  and  certification.  Each  category  of 
savings  was  computed  to  determine  overall  savings/cost  avoidance,  assuming  approval  of 
the  full  funding  requirement,  for  a  total  of  $912.4M  over  the  13-year  program. 

The  Army’s  projected  cost  for  CW  conversion  totals  $268.9M.  Thirty-one  courses  are 
projected  for  annual  conversion  from  1998  through  2002.  The  cost  of  this  conversion 
ranges  from  $426.5K  to  $690.2K  per  course,  with  an  average  of  $594.4K  per  course. 
From  2003  through  2009,  forty-seven  courses  are  projected  for  annual  conversion.  The 
cost  of  this  conversion  is  estimated  from  $212.6K  to  $760.8K  per  course,  with  an  average 
of  $523.4K  per  course.  During  2010,  which  is  the  thirteenth  and  final  year  of  the 
program,  the  last  forty-one  courses  are  projected  for  conversion,  with  a  cost  of  $1 12.4K 
per  course. 
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If  the  Army  accelerates  its  current  DL  program,  Total  Army  -  Active,  Guard,  Reserves  - 
benefits  will  be  realized  in  both  available  dollars  for  reinvestment  and  Soldier  man-years 
(MY)  for  mission  requirements.  Research  has  shown  that  transforming  a  course  into  a 
mix  of  RL  and  DL  applications  will  yield  30%  savings  in  student  instructional  time.  The 
Panel  has  employed  this  DL  estimate  from  research  with  a  conservative  per  diem  factor 
for  cost  and  MY  formulas: 

Per  Diem  cost  avoidance: 

$  =  (RL  hrs.)  x  (.3  /  36  hrs  per  wk.)  x  (7  days)  x  ($30  per  day)  x  (ATRRS  quotas) 

Man  Year  savings: 

MY  =  (RL  hrs.)  x  (.3/36  hrs  per  wk.  /  52  wks.)  x  (ATRRS  quotas) 

(Where  ATRRS  quotas  are  annual  throughputs  from  the  Army  Training  Resource 
Requirement  System  (ATRRS)). 

As  shown,  a  $1 14M  per  year  offset  against  up  front  investment  is  possible  if  the  program 
is  accelerated  to  achieve  optimum  efficiencies.  Additionally  10,532  MYs  would  be 
avoided  and  available  to  unit  commanders  for  mission  requirements. 

The  conversion  of  CW,  if  accelerated  to  maximize  least  cost,  highest  payoff  courses,  will 
yield  Return  on  Investment  (ROI)  in  less  than  three  years  based  on  the  above  mentioned 
formula  (See  Appendix  G  Comparison  Table).  Structure  benefits  will  be  derived  from 
gains  in  manpower  due  to  less  time  away  from  units.  The  Army  used  a  conservative 
projection  of  a  30%  DL  course  reduction  against  student  instructional  time  as  a  model. 
The  Panel  feels  this  is  too  modest  an  assessment  of  the  DL  potential  benefit.  A  higher 
pay  back  may  be  achieved  when  the  savings  in  student  instructional  time  is  determined  to 
be  much  greater  for  selected  courses.  Most  importantly,  the  ripple  effect  in  soldier  MY 
savings  for  mission  performance  will  be  compelling  for  future  force  management 
decisions. 

The  Panel  feels  per  diem  savings  can  offset  the  costs  of  implementing  DL. 
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POTENTIAL  OF  ACCELERATED  PROGRAM  TO^X 
GENERATE  BENEFITS  WITHIN  PLAN  DOLLARSj  \ 


V 


Fiscal  Year 


Distance  Learning 


Potential  of  Accelerated  Program  to  Generate  Benefits  Within  Plan  Dollars 

Given  the  high  rate  of  return  on  the  initial  525  course  DL  capability,  it  is  considered 
imprudent  not  to  aggressively  accelerate  the  rate  of  CW  conversion.  Both  the  ADLP 
proponents  and  the  Panel  agree  that  the  annual  savings  can  exceed  $100  million  once  the 
program  is  in  place.  This  figure  illustrates  some  techniques  that  could  reap  major  savings 
for  the  Army. 

The  lowest  curve  reflects  the  revised  DL  plan  as  of  June  1997;  it  is  included  for 
reference.  The  important  fact  is  that  the  break-even  point  will  not  be  reached  until  2009. 

The  Panel  believes  the  rate  of  CW  development  can  be  accelerated  significantly,  perhaps 
by  as  much  as  a  factor  of  three,  within  the  current  program  budget.  This  acceleration  in 
the  fielding  of  CW  could  result  in  a  return  on  investment  of  nearly  $500M  before  the  year 
2005. 

Finally,  the  Panel  proposes  an  approach  combining  a  CW  "Jumpstart"  with  accelerated 
development  and  leasing  of  the  hardware  to  reduce  the  initial  investment;  this  probably 
represents  an  upper  bound  on  what  the  Panel  hopes  to  achieve.  Jumpstart  has  been 
developed  to  reduce  the  time  and  money  required  to  produce  software;  the  results  are 
major  savings.  The  leasing  of  hardware  reduces  the  initial  investment,  which  allows 
additional  resources  to  be  applied  to  the  CW  conversion  process.  As  reflected  in  these 
recommendations,  the  Panel  recognizes  a  major  change  in  the  approach  to  training  will 
not  be  easy;  therefore,  the  Panel  advocates  the  lease  approach  to  increase  the  Army's 
flexibility  in  the  early  years.  This  could  possibly  move  the  break-even  point  to  the  year 
2003. 
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VISION 


•  Transform  Army  institutional  education  and  training 

FOR  INDIVIDUALS  AND  SMALL  GROUPS  ON  DEMAND 


-  Re-engineering  framework  and  processes 


-  Strengthened  relationship  between  institution  and  field 

bonded  through  common  use  of  CTC  learning  framework  but 
available  asynchronously 


-  Balanced  and  synergistic  employment  of  RL  and  DL  modes  to 
improve  readiness  and  education  together  and  save  time 
and  money 

Distance  Learning  , 


Vision 


The  Panel  recommends  the  Army  take  the  following  actions: 

Transform  its  institutional  education  and  training  to  focus  on  individuals  and  small 
groups  on  demand. 

Reengineer  its  framework  and  processes. 

Strengthen  the  relationship  between  the  institution  and  the  field,  bonded  through  common 
use  of  CTC’s  learning  framework,  but  available  asynchronously.  Education  and  Training 
must  be  a  balanced  and  synergistic  employment  of  RL  and  DL  modes  to  improve 
readiness  and  education  together  and  save  training  time  and  money. 

Derive  DL  solutions  for  it’s  varied  educational  and  training  needs  through  the 
mechanisms  of  partnerships.  Solutions  can  be  integrated  from  contributions  from  three 
classes  of  partnerships  -  Financial,  Technical,  and  Instructional. 

Leverage  non-army  or  DOD  investments. 

Exploit  a  growing  DL  base  for  competitive  purposes. 

Access  world  class  talent  from  industry,  academe  and  government  rather  than  develop  a 
full  or  even  partial  in-house  DL  infrastructure. 


29 


Focus  its  resources  —  people,  technical  capabilities,  and  financial  means  -  on  being  a 
smart  buyer-partner. 

Establish  and  refine  the  intellectual  framework  and  technical  architecture  for  DL  by 
applying  the  best  collaborative  methods  from  industry  and  academe  with  the  CTCs 
advances  prevalent  in  the  Army. 

Provide,  only  where  necessary,  army  unique  solutions. 

Lease  rather  than  buy  to  avoid  the  future  legacy  system  trap  for  the  immediate  near  term. 

Aggressively  pursue  the  use  of  current  and  emerging  DL  capabilities  to  address  its 
current  and  probable  future  challenges. 

Conduct  a  set  of  DL  experiments  based  upon  partnerships  to  establish  benefits,  costs, 
strengths  and  limitations. 

Pursue  the  research  to  make  future  DL  activities  more  effective  and  affordable  using 
commercial  and  academic  sources. 

Through  external  research,  establish  how  people  learn  in  order  to  improve  conceptual, 
cognitive  and  motor  skills  training  in  the  Army. 

Establish  methodologies  and  techniques  to  support  learning  both  individually  and  in 
groups  (small  to  large  settings). 

Be  a  smart  industrial  and  academic  base  builder  and  user  of  DL  capabilities. 
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DL  TRAINING  MANAGEMENT 

Development  of  Individual  Training  Schedule 


Technology 
is  not  a 
limitation 


Distance  Learning 


Individualized , 
asynchronous , 
training  in  units 


DL  Training  Management  -  Development  of  Individual  Training  Schedule 

Today  the  development  of  an  individual’s  training  schedule  is  almost  entirely  based  upon 
the  availability  of  scheduled  schoolhouse  courses. 

With  DL,  the  training  schedule  will  be  based  upon  the  Soldier’s  availability  and  is 
independent  of  the  Soldier’s  duty  assignment  location.  The  key  is  to  design  the  courses 
recognizing  technology  is  not  a  limitation.  Ultimately,  the  Soldier  should  be  able  to  gain 
all  needed  education  and  training  (except  socialization  and  soldierization)  through 
inexpensive,  ubiquitous  receptors  (TV  sets,  PC’s  etc.)  accessing  the  course  material 
through  a  variety  of  channels  (satellite  downlinks,  internet,  snail  mail,  et  al).  The  course 
material  may  resemble  present  courses,  but  it  could  well  include  CTC’s,  vignettes  and 
access  to  digital  ‘libraries’  to  permit  a  construction  of  training  relevant  to  the  individual 
Soldier’s  assignments.  Several  of  these  modalities  will  allow  student  collaboration, 
asynchronous  contact  with  instructors,  and  Intelligent  Tutors  programmed  into  the  course 
material. 

In  the  not  too  distant  future,  the  following  scenario  might  take  place. 

A  soldier  in  a  unit  is  told  to  complete  four  segments  of  a  course  to  qualify  as  a  squad 
leader. .  Over  the  next  six  weeks,  the  Soldier  only  has  certain  days  and  hours  available. 
The  Soldier’s  first  action  is  to  query  the  PC  to  find  out  which  of  the  four  segments  he  can 
diagnostically  pretest  and  successfully  pass  without  taking  those  particular  segments. 
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The  Soldier’s  available  days  and  time  for  the  next  six  weeks  are  then  input  into  the 
computer,  and  the  following  questions  are  asked: 

Of  the  remaining  segments,  what  is  going  to  be  available  on  the  net  as  formal  instruction 
while  I  am  available  to  view  and  participate? 

Of  the  remaining  segments,  what  is  going  to  be  available  on  the  net  as  archive/library 
information  while  I  am  available  to  view  and  participate? 

Of  the  remaining  segments,  what  elements  are  available  in  CBI  with  interactive  tutors? 

Are  any  other  soldiers  in  my  unit  taking  any  of  the  four  segments  and  the  elements  that  I 
am  lacking? 

Are  there  subject  matter  experts  (SMEs)  in  my  unit?  If  not,  are  any  SMEs  going  to  be 
available  over  the  net  or  by  cable  during  my  available  times? 

Of  the  material  not  declared  available  in  my  previous  queries,  is  any  of  it  available  in  the 
‘Army  Yellow  Pages’? 

Of  the  material  not  declared  available  in  my  previous  queries,  are  there  equivalent 
elements  available  through  other  DoD,  industrial,  or  civilian  feeds  that  will  permit  me  to 
qualify? 

All  of  these  actions  are  well  within  the  realm  of  possibility.  The  Soldier  has  prepared  his 
training  schedule  and  is  ready  to  “roll”  in  less  than  an  hour.  There  is  no  need  for  the 
Soldier  to  leave  the  unit  for  an  extended  period  of  RL. 
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CONCLUSION 

Technology  will  not  limit  DL  implementation 
neither  will  network  access  or  services 

WHY? 


•  Cost-performance  of  combined  building  blocks 
(processors,  memory,  etc.)  will  improve  by  a 
factor  of  1000  by  2010 


Commercial  global  SATCOM  access  and  costs 
will  be  competitive  with  fiber 


2010  Projection 

The  implementation  of  DL  will  not  be  limited  by  technology,  and  its  evolution  will  not  be 
obstructed  by  network  access  or  services.  The  reason  is  that  cost-performance  of 
combined  building  blocks  (processors,  memory,  etc.)  will  improve  by  a  factor  of  one 
thousand  by  the  year  2010,  and  commercial  global  SATCOM  access  and  costs  will  be 
competitive  with  fiber. 

The  speed  and  capacity  of  computer  and  communications  systems  has  been  increasing 
exponentially  for  over  thirty  years.  During  the  decade  of  1985  to  1995,  the  overall 
performance  of  desktop  computers  increased  by  more  than  a  factor  of  fifty  in  both  speed 
and  random-access  memory  capacity,  both  measures  doubling  approximately  every 
fifteen  to  twenty-four  months.  During  the  same  period,  the  cost  of  desktop  computers 
steadily  decreased,  going  down  by  half  every  two  years.  Although  there  are  physical 
limitations  to  overcome  in  the  future,  current  technology  trends  indicate  this  exponential 
growth  rate  will  continue  well  past  the  year  2005.  At  this  rate,  the  desktop  system  of 
2010  will  be  three  orders  greater  in  magnitude  than  current  machines.  Given  these 
trends,  it  is  clear  the  power  of  computing  technology  will  not  limit  the  implementation  of 
any  DL  plan.  Furthermore,  any  such  plan  must  assume  a  process  of  continual 
redevelopment  and  redeployment  is  required  to  fully  utilize  the  increasing  power  of 
computing  technology. 

As  with  computers,  telecommunications  technology  has  been  advancing  exponentially  in 
terms  of  network  bandwidth.  Industry  and  the  military  are  building  world-wide 
networks,  with  dense  fiber  networks  available  across  the  CONUS  by  the  year  2000  and 
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world-wide  high-speed  networking  available  early  next  century.  In  the  military,  forces 
will  be  supported  by  a  variety  of  global  networks,  including  the  MILSATCOM,  the 
Tactical  Internet,  and  wide-band  global  broadcast  satellite  capability.  The  commercial 
sector  is  also  deploying  worldwide  high-speed  networks,  with  over  ten  global  commercial 
satellite  systems  currently  under  development.  These  include  Iridium  (Motorola), 
Teledesic  (Gates-McCaw),  Globalstar  (Loral/Alcatel),  and  others.  In  all  cases  in  the 
commercial  sector,  the  expectation  is  that  global  satellite  communications  costs  will  be 
competitive  with  decreasing  costs  in  terrestrial  fiber-optic  service  and  pricing  will  be 
proportional  to  the  amount  of  bandwidth  utilized.  Given  these  trends,  it  is  clear  that  the 
availability  of  high-speed,  worldwide,  wide-band  telecommunications  will  not  be  a 
limiting  factor  in  the  implementation  of  any  DL  plan. 
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A  SUMMING  UP 


•  A  SUCCESSFUL,  WORLD  CLASS  DL  EDUCATION  CAPABILITY  IS  CRUCIAL 
FOR  TODAY’S  AND  TOMORROW’S  ARMY.  ITS  REALIZATION  MUST  BE  GIVEN 
HIGH  PRIORITY  -  JUST  AS  THE  ARMY  HAS  DONE  WITH  ITS  CTC’S. 

•  THE  ARMY  SHOULD  SET  FOR  ITSELF  WORLD  CLASS  OBJECTIVES  FOR 
READINESS,  AND  EDUCATIONAL  LEVELS  ACHIEVED  WITH  FISCAL 
EFFICIENT  DL.  THEY  SHOULD  EQUAL  OR  EXCEED  THOSE  ACHIEVED  BY 
INDUSTRY  AND  EDUCATION  WITH  COLLABORATIVE  ASYNCHRONOUS 
METHODS.  (THE  ARMY  LEADS  THE  WORLD  IN  COLLABORATIVE 
SYNCHRONOUS  METHODS.) 

•  IT  IS  THE  ASB  VIEW  THAT  PROGRAM  OBJECTIVES  COULD  BE: 

-  MUCH  REDUCED  TURBULENCE  AND  ENHANCED  READINESS 


-  AT  LEAST  A  “ONE  SIGMA”  IMPROVEMENT  IN  STANDARDS  ACHIEVED 


V 


-  TTHS  MANPOWER  AND  TDY  SAVINGS  OF  >50%  FROM  MUCH  REDUCED 
COURSE  LENGTHS  AND  IN-UNIT  AND  HOME  STATION  LEARNING. 


Distance  Learning 


A  Summing  Up 

Distance  Leaning  will  fulfill  the  Army’s  need.  A  robust  DL  capability  will  provide  great 
benefits  for  the  Army  regarding  unit  readiness  and  Soldier  education  as  well  as  save  time 
and  money.  Both  the  ADLP  and  the  National  Guard  program  designs  focus  on  a 
synchronous  remote  DL  plan  based  on  an  RL  intellectual  framework.  Upgrading  the 
ADLP  from  its  inception  until  now  has  not  changed  the  basic  tenets  of  the  plan.  The  base 
concepts  of  the  ADLP  were  experimentally  validated  six  to  eight  years  ago.  Reviewing 
the  changes  in  DL  technology  over  the  same  period,  the  Panel  found  major  happenings  in 
information  technology.  During  this  period,  there  was  an  explosive  growth  of  the 
Internet.  Additionally,  the  rapid  expansion  of  fiber  optic  cable  resulted  in  cable  available 
to  every  high  school  and  college  in  many  states.  The  formation  and  operation  of  on-line 
university  courses  and  the  creation  of  an  educational  technology  industrial  base  was 
evolving.  Asynchronous  DL-collaborative  programs  have  emerged  in  the  last  five  years. 
The  Panel  also  discovered  many  programs  for  teachers  to  leam  how  to  enhance  education 
by  utilizing  DL  facilities.  During  this  same  period,  the  Army  lost  most  of  its  training 
developer  personnel. 
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A  SUMMING  UP  (CONTINUED) 


THE  CURRENT  DL  PLAN  IS  AN  ADEQUATE  STARTING  POINT 


•  THE  FUTURE  TIME  PHASED  DL  PLAN  VERSIONS  SHOULD  REFLECT  A 
STATEMENT  OF  THE  OBJECTIVE  STATE  AND  THE  RELATIONSHIP  TO 
EVOLVING  ARCHITECTURE  FOR  A  BALANCED  PROGRAM  OF  RL  AND 
SYNCHRONOUS  AND  ASYNCHRONOUS  COLLABORATIVE  DL  EMPLOYING 
COMMON  STANDARDS  AND  APPROPRIATE  COURSEWARE 


V 


THE  PLAN  SHOULD  REFLECT  PERFORMANCE  OBJECTIVES  (AS  PREVIOUSLY 
MENTIONED)  AND  RELATED  COSTS  AND  SAVINGS 

THE  PLAN  SHOULD  INCLUDE  A  VISIONARY  FINANCIAL  STRATEGY  WHICH 
LEVERAGES  EXTERNAL  RESOURCES,  CREATES  SAVINGS  INSIDE  THE  ARMY, 
AND  SEIZES  LEADING  EDGE  EDUCATIONAL  OPPORTUNITIES 


Distance  Learning 


The  Panel  realizes  the  CW  development  challenges  were  addressed  when  TRADOC 
recognized  the  consequences  of  not  satisfying  the  demands  and  orchestrated  a 
community-wide  effort  to  set  priorities.  As  this  Panel's  report  was  briefed  in  June  of 
1997,  525  out  of  2500  courses  have  been  identified  as  targets  for  modernization  for  DL 
delivery.  ADLP  and  TATS-C  are  addressing  CW  unification  and  overall  DL  priorities. 
CW  development  is  judged  to  be  the  most  important  issue  facing  the  Total  Force. 
Conversion  is  planned  to  take  thirteen  years.  Accelerating  CW  development  and  fielding 
by  a  factor  of  2  to  3  puts  the  entire  program  on  a  self-paying  basis  in  three  to  five  years, 
rather  than  the  current  plan  of  eleven  to  thirteen  years. 
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ASB  JUDGMENT  ON  WHAT  TO  DO 
(STATUS  -  26  June  1997) 


-T\ 


Program 

•  Jump-start  a  two  track  program  now  (FY‘98) 

-  Using  synchronous  remote  classrooms 

-  Adding  collaborative  asynchronous  capabilities  at  the  same  time 

•  Use  low-end  technology  for  remote  classrooms  and  one-way  video 

-  To  greatest  extent  possible.  Lease  and  partner  to  save  money. 

•  Allow  for  two-way  video  and  high-end  classrooms  to  underwrite 

-  Use  of  simulations  as  learning  aids 

-  Operational  planning,  exercising  and  rehearsal 

-  Other  just-in-time  needs  which  are  contingency  driven 
Courseware 

•  Accelerate  courseware  conversion  by  a  factor  of  three.  Use  savings  from 
above 

•  Focus  conversion  in  high  payoff  areas  with  greatest  student  throughput-far 
minimum  courseware  needed.  Likely  candidates  are: 

-  Skill  sustainment  (AC  &  RC) 

-  Reclassification  (principally  RC) 

Create  capabilities  to  develop  and  produce  high  payoff  courseware  for 
individualized  and  collaborative  use 


STATUS  (at  publication) 

This  is  the  Current 
Program  Strategy. 


The  June  97  Plan  is  the 
Evolutionary  starting 
Point 


V 


Situation  improved,  but 
courseware  is  still  the 
“Long  and  Critical  Tent 
Pole." 


Distance  Learning 


The  Panel’s  Judgment  On  What  to  Do 

The  Panel  envisions  a  three-phased  approach  to  DL  modernization  where  the  following 
occurs: 

Jump-start  a  two-track  program  in  FY  ’98  using  synchronous  remote  classrooms  and 
adding  asynchronous  collaborative  capabilities  as  time  &  funding  permit. 

Use  low-end  technology  for  remote  classrooms  and  one-way  video  to  the  greatest  extent 
possible,  while  leasing  and  partnering  to  save  money. 

Allow  for  two-way  video  and  high-end  classrooms  to  underwrite  the  use  of  simulations 
as  learning  aids;  for  operations  planning,  exercising  and  rehearsal;  and,  other  just-in- 
time  contingency  driven  requirements. 

Accelerate  CW  conversion  by  a  factor  of  three.  Use  savings  from  per  diem  accounts 
where  soldiers  remained  on  station  and  completed  courses.  Focus  conversion  in  high 
payoff  areas  with  greatest  student  throughput  for  minimum  CW  needed.  Likely 
candidates  are  skill  sustainment  training  for  AC  and  RC,  and  reclassification  training 
primarily  for  the  RC.  Create  capabilities  to  develop  and  produce  high  payoff  CW  for 
individualized  and  collaborative  use. 

The  Panel’s  judgments  are  based  on  the  fact  comparable  civilian  training  is  the  easiest  to 
remote  because  only  a  few,  if  any,  training  aid  devices  are  required.  Thus  the  actual  RL 
course  may  be  offered  by  remote  delivery  with  little  investment.  The  other  factor  is  that 
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sustainment  and  reclassification  training  have  such  a  significant  demand  that  a  30% 
reduction  in  RL  course  length  would  result  in  notable  savings  in  travel,  per  diem,  and 
readiness. 

The  Army  should  experience  significant  benefit  through  DL  applications  in  skill 
sustainment  for  both  the  active  and  reserve  components:  MOSQ  reclassification  training, 
primarily  for  the  reserve  components,  and  just  in  time  training  needs  of  the  total  army. 
Samplings  indicate  student-teacher  ratios  could  be  raised  from  the  Army’s  present  fifteen 
to  one  to  the  industry  model’s  fifty  or  one  hundred  to  one.  DL  should  also  reduce 
monitoring  requirements  and  result  in  dollar  savings  and  Soldier  MY  returns  from  TTHS 
to  unit  mission  requirements.  Synchronous  applications  could  be  satisfied  through  low 
cost,  low  end,  low  bandwidth,  one-way  video  classroom  set-ups.  Internet  &  Chat  room 
type  links  for  students  should  be  aggressively  pursued. 

To  maximize  learning,  minimize  dollar  and  unit  turbulence  costs,  and  retain  its  leading 
edge,  the  DL  program  has  to  meet  certain  criteria.  The  program  must  be  based  on  an 
intellectual  framework  that  allows  tradeoff  decisions  to  maximize  learning.  This 
framework,  discussed  earlier,  consists  of  students,  whose  prior  knowledge,  learning  skills 
and  motivation  must  be  taken  into  account;  learning  objectives;  and  enablers 
(technology).  Assessment  of  this  framework  will  support  practical  decisions  about  which 
technologies,  facilities  and  media  will  maximize  learning  and  minimize  cost. 

Maximizing  the  cost-effectiveness  of  training  over  time  cannot  be  achieved  with 
marginal  changes  to  the  existing  program.  The  program  has  to  be  completely  re¬ 
engineered  if  it  is  to  achieve  the  learning  gains  that  research  shows  can  be  realized. 
Minimizing  the  unit  turbulence  attributable  to  RL,  which  will  in  turn  minimize  the 
investment  and  the  recurrent  costs,  can  do  this.  Re-engineering  is  a  process,  not  a  single 
event,  and  the  program  must  be  repositioned  to  handle  technological  obsolescence  and 
seize  emerging  opportunities. 

Finally,  achieving  program  objectives  must  be  accelerated  to  provide  earlier  savings. 

TRADOC  feels  the  Panel’s  recommended  program  is  the  current  strategy  under  their 
June  97  ADLP  update  and  their  document  is  an  evolutionary  starting  point.  Additionally 
TRADOC  feels  the  CW  situation  has  improved,  but  agrees  it  is  the  critical  issue  to 
constantly  monitor. 
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RECOMMENDATION  1 


Action:  Formulate  Department  of  Army  DL  Plan  to  transform 
learning  using  institutional  processes 

Who:  VCSA  with  CG  TRADOC,  and  AAE 

What:  Appoint  an  Architect  support  with  experts  from  all 
involved  organizations 

When:  Charter  architect  by  31  Aug  97. 

Have  architect  formulate  plan  by  30  Oct  97. 

Complete  DA-level  review  by  1 5  Dec  97 
Continue  semi-annual  review  by  AVCSA  and  DCG 
TRADOC 


•CG  TRADOC  is 
Program  Architect 

•Army  CIO  is  Technical 
Architect 

•Project  Office 
Established  with: 

-PM  from  ST  AMIS  & 
working  RC  counterparts 

-TP  10  within  TRADOC 
DCST 

•Evolutionary  Plan 


v 


BENEFIT:  DL  is  institutionalized  -  “Put  on  right  Army  footing " 


Distance  Learning 


Recommendation  1 


The  use  of  the  word  ‘architect’  is  deliberate.  It  calls  attention  to  the  Panel’s  belief  a 
single  person  needs  to  be  responsible  for  the  manner  in  which  the  elements  are  arranged 
and  organized. 

Without  the  appointment  by  CG  TRADOC,  the  AAE,  and  the  VCS  of  a  single  architect, 
the  Panel  believes  the  time  frame  proposed  above  cannot  be  met. 

Further,  once  the  plan  has  been  constructed,  approved  by  the  Army,  and  funded,  the 
architect  would  step  aside  in  favor  of  normal  Army  implementation. 

Since  development  of  this  report  and  the  recommendations,  DA  &  TRADOC  have 
initiated  the  following  actions: 

CG  TRADOC  is  the  Program  Architect. 

The  Army  CIO  is  the  Technical  Architect. 

A  Project  Office  has  been  established  at  TRADOC  with  the  Program  Manager  (PM)  from 
ST  AMT.S  and  working  RC  counterparts.  The  TRADOC  Program  Integration  Office 
(TPIO)  is  within  the  TRADOC  DCST. 

The  Total  Army  Distance  Learning  Program  (TADLP)  Master  Plan  (MP)  is  the 
evolutionary  document  being  updated  by  TRADOC. 
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RECOMMENDATION  2 


Action:  Obtain  resources  to  “Jump-start”  overall  program 
Who:  AAE  and  ASA(FM&C)  (including  NG  as  well) 

What:  Appoint  a  DL  Chief  Financial  Officer  (CFO) 

When:  Start  by  31  Aug  97  activities  to: 

-  Find  partnerships 

-  Obtain  funding  outside  the  Army 

-  Obtain  up-front  and  continuing  resources  from  industry 

-  Obtain  in  kind  resources 

Distance  Learning 


Recommendation  2 


The  Panel’s  second  recommendation  is  to  suggest  the  appointment  of  a  DL  Chief 
Financial  Officer  (CFO).  Although  an  unusual  position  for  the  Army,  the  function  of  the 
CFO  would  be  to  establish  partnerships,  both  financial  and  in-kind,  with  other  non-Army 
agencies  (e.g.,  DARPA  or  Department  of  Education)  in  order  to  create  a  financial  win- 
win  situation.  The  Panel  expects  that  although  the  program  will  pay  for  itself, 
particularly  in  the  low  risk  synchronous  mode,  there  will  be  a  need  for  additional 
resources  to  “Jumpstart”  the  overall  program  and  minimize  the  unknown  risks  of 
asynchronous  modes.  Thus  this  position  will  support  the  timely  execution  of  the 
program.  The  recommended  action  officer  is  the  Army  Acquisition  Executive  (AAE) 
supported  by  Assistant  Secretary  of  the  Army,  Financial  Management  and  Comptroller 
(ASA  FM&C). 
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RECOMMENDATION  3 


Action:  Jump-start  DL  Program  with  In  Process 

-  Low  Risk  and  Low  End  Remote  DL  elements 

-  “Best  of  Breed”  courseware  for  high  payback  usage 

-  Low  Risk  collaborative  elements 

Who:  CG  TRADOC  and  AAE 

What:  Accelerate  TATS-C  to  provide  priority  courseware 
Rapidly  clean  up  and  focus  priority  courseware 
Field  DL  remote  and  available  collaborative  DL  elements 

When:  Start  by  mid  August  97 

Formulate  schedule  by  30  Sept  97 


V 


Distance  Leamil 


BENEFIT:  Improve  readiness  and  start  early  savings  stream 


Recommendation  3 


The  Army  faces  a  major  task  in  implementing  the  ADLP.  As  identified  in  previous 
recommendations,  the  transition  to  DL  will  engender  major  changes  in  the  manner  in 
which  the  Army’s  education  process  functions.  Despite  these  uncertain  conditions,  the 
major  cost  savings  produced  by  DL  argue  for  an  aggressive  approach  to  the 
implementation  of  the  DL  environment. 

The  Panel  has  examined  the  proposed  plan  for  CW  conversion  and  believes  the  process 
can  be  accelerated  substantially.  By  emulating  the  Jumpstart  processes  developed  for 
rapid  software  development,  and  by  selecting  the  best  available  CW,  the  Army  can 
significantly  reduce  the  time  and  level  of  effort  required  to  field  an  initial  DL  capability. 
The  dynamics  of  DL  technology  and  the  rapid  progress  being  made  in  the  “art”  of  DL 
argue  for  a  modest  initial  capability,  fully  recognizing  future  courses  and  technology  may 
bear  little  resemblance  to  the  initial  capability.  The  critical  factor  is  early  implementation 
to  produce  rapid  cost  savings  to  be  invested  in  future  DL,  apportioned  as  required. 

The  CG  TRADOC  should  accelerate  the  TATS-C  process  to  support  rapid  conversion  to 
DL.  CG  TRADOC  and  the  AAE  must  reduce  the  major  investment  being  made  in 
hardware,  which  will  be  obsolete  within  a  matter  of  years.  It  is  critical  the  selected 
enablers  minimize  the  initial  investment  and  yet  will  not  introduce  a  major  expense  in  the 
out  years  as  the  equipment  reaches  the  end  of  its  service  life. 
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The  importance  of  this  activity  argues  for  an  early  start  in  the  planning  process  and  the 
need  for  a  schedule  of  execution  by  the  start  of  the  Fiscal  Year. 

Smart,  aggressive  execution  of  the  CW  conversion  coupled  with  a  reasoned  hardware 
investment  can  reap  rapidly  improved  readiness  and  achieve  substantial  savings. 

TRADOC  feels  this  recommendation  is  in  process. 
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RECOMMENDATION  4 


V 


Action:  Establish  a  DL  Design  Bureau  to  transform 
processes  and  organizations  to  lead  Army 
into  the  asynchronous  collaborative  DL  world 


Who: 

What: 


When: 


CG  TRADOC 

Establish  a  small  team  of  Army,  University 
and  Industry  experts  to  design,  experiment, 
test  and  define  the  RL  &  DL  framework 

Initiate  by  1  Oct  1997,  Provide  following 
described  on  next  page 


TRADOC 
has  declared 
their  intent 
to  implement 
an  alternative 
methodology  and 
management 
approach 


Distance  Learning 


Recommendation  4 


The  purpose  of  this  recommendation  is  to  create  a  DL  Design  Bureau  with  the  mission  to 
provide  command  emphasis  for  the  DL  transformation  and  provide  the  processes  and 
sources  of  expertise  for  developing  standards,  for  example  CW,  for  the  asynchronous  DL 
environment.  It  is  recommended  the  CG  TRADOC  establish  a  team  of  experts  from  the 
Army,  universities,  and  industry  who  will  facilitate  the  design  and  assessment  for  re¬ 
engineering  the  DL  education  process. 
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RECOMMENDATION  4  (continued) 


When:  Initiate  by  1  Oct  97 

At  ONE  year  (1  Oct  98),  provide 

•  Approved  plan  for  transforming  regulations,  processes,  and 
organizations  to  support  the  DL  framework 

•  Methodology  for  centralized  courseware  design  and  assessment 

•  Procedures  for  decentralized  development  of  course  content 

•  Procedures  for  smart,  centralized  courseware  contracting,  with 
emphasis  on  best  value  and  cost/benefit 

•  Plan  for  jump-starting  Army  DL  courseware  development  capability  - 
selection  and  training,  courseware  developers  to  make  them  smart 
buyers/developers 

•  At  THREE  years  (1  Oct  00),  DL  framework  should  be  fully  operational 
Coordinate  closely  with  DL  Futures  Laboratory 

Distance  Learning 


The  Panel  recommends  the  Design  Bureau  be  set  up  by  1  Oct  97.  Sample  activities 
would  be  to  create  a  methodology  for  centralized  design  of  CW  and  decentralized 
development.  It  is  expected  the  vast  majority  of  CW  will  be  contracted  out.  Another 
example  of  processes  needing  change  is  the  selection  of  training  for  training-developers 
to  make  them  smart  buyers/developers.  The  Bureau  should  coordinate  closely  with  the 
organization  discussed  in  the  next  recommendation  (The  DL  Futures  Laboratory)  to 
facilitate  the  implementation  of  ideas  identified  by  the  Laboratory. 

TRADOC  has  declared  their  intent  to  implement  an  alternative  methodology  and 
management  approach. 
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RECOMMENDATION  5 


Action:  Create  a  distributed  DL  Futures  Lab  (Consortia) 

To  explore  new  concepts  for 

•  Effective  training  and  learning  methods  and  technologies 

•  Application  of  technology,  management  of  obsolescence  and  affordable 
strategies 

•  Courseware  design,  development,  and  evaluation 
Direct  external  research  on  how  soldiers/adults  learn 
Virtual  connection  with  Centers  of  Excellence 

Who:  CG  TRADOC  (Assisted  by  AAE) 

What:  Task  DCG  CA  TRADOC  to  establish  charter,  form  the 

group,  and  establish  DL  consortia  with  civilian  and 
emerging  opportunities 

When:  Initiate  action  by  1  Oct  97 

Coordinate  with  DL  Design  Bureau 

Use  Integrated  Product  Team  (IPT)  and  Integrated  Concept  Team  (ICT) 

Distance  Learning 


BENEFIT:  Long-term  DL  vision,  minimize  surprises 
or  lost  opportunities 


Recommendation  5 

The  rapid  evolution  of  the  entire  field  of  DL,  both  how  a  student  learns  and  the 
technology  to  support  DL,  demands  a  process  whereby  the  Army  monitors  and  exploits 
these  capabilities.  The  Panel  recommends  the  establishment  of  an  Army  led  consortium 
to  harvest  the  beneficial  elements  from  this  field.  This  consortium  should  be  viewed  as  a 
laboratory  with  distributed  networked  members.  Since  much  of  the  progress  is  being 
made  within  the  academic  community,  it  is  critical  the  laboratory  incorporates  leading 
universities  as  well  as  commercial  training  organizations  as  represented  by  the  global 
activities  of  the  automotive  manufacturers. 

The  lab  would  be  expected  to  explore  training  and  learning  methods  as  well  as  their 
supporting  technologies.  Given  the  rapid  rate  of  change  in  the  technology  area,  methods 
of  reducing  the  burden  of  obsolescence  should  be  a  matter  of  priority.  The  techniques 
and  perhaps  technology  to  support  the  design  and  development  of  CW  can  pay  big 
dividends  by  providing  an  economical  means  of  producing  and  updating  future  training 
needs. 

The  Panel  suggests  the  laboratory  be  a  TRADOC  activity  with  strong  support  from  the 
AAE.  It  is  critical  those  elements  of  the  acquisition  community  who  produce  “new 
equipment  training”  be  a  party  to  this  process.  This  combined  Army,  Civilian,  university 
and  commercial  venture,  operating  under  the  leadership  of  the  DCG  CA  TRADOC,  can 
and  should  lead  the  way  in  the  smart  efficient  application  of  new  technology  to  the 
Army’s  continuing  CW  load. 
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The  Panel  suggests  the  lab  be  initiated  by  1  Oct  97,  with  the  expectation  it  might  be 
producing  useful  insights  by  early  98.  The  operation  of  this  group  should  lean  heavily  on 
the  lessons  learned  from  the  operation  of  both  ICT  and  IPT’s.  This  team  approach  to 
problem  solving  has  proven  itself  to  be  a  powerful  tool  in  focusing  on  the  high  payoff 
strategies  in  system  development.  It  can  be  applied  with  similar  effect  to  the  training 
problem. 

The  mission  of  this  group  must  be  to  ensure  the  training  community  does  not  meet  with 
unpleasant  surprises  and  effective  opportunities  are  not  missed. 
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RECOMMENDATION  6 


Action: 

Who: 


Add  a  DL  program  element  for 
high-payoff  DA  Civilian  education 


Army  Acquisition  Executive 


TRADOC  has  undertaken 
Program  for  ASA  M&RA 


What:  Define  DL  initiatives:  Alpha  contracting,  environmental 
issues,  market  surveys,  COTS  procurement,  et  al. 
Requirement  and  sources  for  funding 


When:  1  October  97  start  date 


V 


BENEFIT:  Benefits  of  DL  for  uniformed  members  extended  to  DA  Civilians 


Distance  Learning 


Recommendation  6 


The  Panel’s  final  recommendation  is  to  add  a  program  element  for  DL  to  train/educate 
Department  of  Army  civilians.  The  current  Army  DL  program’s  main  focus  is  on  service 
members.  The  benefit  of  DL  (e.g.,  improve  performance,  decrease  training  time)  for 
uniformed  members  would  be  extended  to  DA  civilians.  Suggested  DL  initiatives  might 
involve  alpha  contracting  or  environmental  issues.  The  suggested  action  agency  is  the 
Army  Acquisition  Executive  (AAE). 

TRADOC  has  undertaken  a  program  for  civilian  DL  with  the  Assistant  Secretary  of  the 
Army,  Manpower  and  Reserve  Affairs  (ASA  M&RA). 
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Distance  Learning  -  Summer  Study 
Summary  Observations 


•  Army  has  a  Plan.  It’s  good  for  openers;  get  an  Architect, 

a  Driver,  $$  and  move  out. 

•  Army  has  world  class  framework  model:  map  the  CTC 

into  schoolhouse  and  leverage  DL  leading  edge 

•  Re-engineer  framework  to  emphasize  asynchronous 

collaborative  mode 

•  Universities  and  industry  can  be  a  real  help 

•  Technology  is  not  a  limiting  factor 

•  Major  savings  in  TTHS  and  infrastructure 


^  Be  all  that  you  can  be. 

Distance  Learning  _ _ _ _ 


Distance  Learning  -  Summer  Study  Summary  Observations 

The  ADLP  is  not  only  a  start  but  also  the  lead  document  in  the  entire  DOD.  The  Army 
should  appoint  the  architect,  put  a  clear  vision  in  place  with  the  future  lab,  design  bureau 
and  CFO,  to  drive  the  program;  then  aggressively  seek  out  funding  sources  other  than  the 
Army.  “Accelerate  Distance  Learning”! !  Use  the  Army’s  CTC  framework  to 
incorporate  industry  and  academe  DL  leading  technologies  to  leverage  the  very  best 
available  innovations.  Asynchronous  applications  will  require  a  major  re-engineering 
effort,  and  the  emphasis  should  produce  acceptable  levels  of  DL  collaborative 
environments.  Partner,  collaborate,  and  form  consortiums  to  take  maximum  advantage  of 
developments  from  Universities  and  industry  (Ford  &  One  Touch).  The  Army  should 
move  to  realize  major  savings  in  TTHS  and  infrastructure  through  a  dedicated,  more 
centralized  effort  in  CW  development. 

A  world  class  DL  education  capability  is  crucial  for  today’s  Army.  The  Army  must  set 
world  class  objectives  for  both  readiness  and  unsurpassed  DL  educational  enhancements; 
objectives  exceeding  those  of  industry  &  academe.  Program  objectives  of  readiness, 
reduced  turbulence,  improved  standards,  cost  avoidance  and  man-years  returned  to  units 
are  all  achievable.  The  TADL  MP  must  evolve  to  phased  versions  with  a  clear  objective 
state,  relationship  to  evolving  architecture,  balanced  RL,  synchronous  and  asynchronous 
DL-Collaborative  employing  common  standards  and  CW.  The  plan  with  stated 
performance  objectives  and  related  costs  and  savings  should  include  a  visionary  financial 
strategy. 
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APPENDIX  A 


TERMS  OF  REFERENCE 
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DEPARTMENT  OF  THE  ARMY 
OFFICE  OF  THE  ASSISTANT  SECRETARY 
RESEARCH  DEVELOPMENT  AND  ACQUISITION 
103  ARMY  PENTAGON 
WASHINGTON  DC  20310-0103 

7  MAR  1997 


Dr.  Michael  S.  Frankel 
Chair,  Army  Science  Board 
Research,  Development  and  Acquisition 
103  Army  Pentagon 
Washington,  DC  20310-0103 

Dear  Dr.  Frankel: 

I  request  that  you  conduct  an  Army  Science  Board  (ASB)  Summer  Study 
on  "Distance  Learning."  The  assessment  should  address,  as  a  minimum,  the 
Terms  of  Reference  (TOR)  described  below.  The  ASB  members  appointed 
should  consider  the  TOR  only  as  guidelines  and  may  include  in  their  discussions 
related  issues  deemed  important  or  suggested  by  the  sponsor.  Modifications  to 
the  TOR  must  be  coordinated  with  the  ASB  Office. 

Background. 

a.  The  Army  faces  a  massive  and  unique  set  of  education  and  training 
challenges.  The  Army  is  geographically  dispersed  with  units  at  well  over  a 
thousand  locations.  Personnel  turnover  rates  can  reach  as  high  as  20  to  40 
percent  for  given  military  occupational  specialties.  New  equipment,  doctrine  and 
techniques  are  continually  infused  to  upgrade  and  enhance  unit  capabilities  in 
order  to  support  the  wide  range  of  missions  the  Army  must  be  able  to 
accomplish.  These  factors  taken  together  mean  that  education  and  training  is  a 
pervasive,  continual  process  which  consumes  a  huge  amount  of  resources. 

b.  It  is  difficult,  if  not  impossible,  to  find  an  analogous  set  of 
circumstances  elsewhere  in  government,  industry  or  academia  that  rival  the 
challenges  the  Army  confronts  in  meeting  educational  and  training  requirements 
and  demands. 

c.  Traditionally,  Army  education  and  training  has  been  characterized  by 
the  use  of  fixed  facilities  and  movable  populations  of  Soldiers.  The  Army  along 
with  other  world  class  organizations  are  “learning"  as  well  as  “doing” 
organizations.  Learning  is  part  of  continuous  improvement  and  maintenance  of 
competitive  skills.  The  classic  schoolhouse  delivery  solutions  are  no  longer 
appropriate  except  in  special  situations. 

d.  At  the  same  time,  the  Army  has  been  in  the  forefront  of  advancing  new 
concepts  and  adapting  new  technologies  for  educating  and  training  its  Soldiers. 

It  is  the  DOD  lead  Service  for  advanced  distributed  simulation  which  could  be 
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considered  as  training  counterpart  of  distance  learning.  It  has  pioneered 
interactive  and  embedded  training  in  the  same  integrated  manner. 

e.  The  Army  and  DOD  are  making  substantial  investments  in  distance 
learning.  The  Army  plans  to  execute  a  significant  distance  learning  program, 
estimated  to  cost  about  one  billion  dollars  over  the  POM.  At  the  same  time, 
industries  and  academic  institutions  are  addressing  some  of  their  needs  with 
distance  learning  methods  and  tools. 

f.  To  date,  virtually  all  the  hardware  and  network  services  (local  and  long 
distance)  have  been  met  using  the  technologies  of  the  ongoing  information 
revolution.  Important  software  applications  and  innovations  derive  from  this 
revolution. 

g.  However,  course  content  which  has  data  base  and  other  software 
components  requires  its  own  research  and  development,  Verification,  Validation 
&  Analysis  and  fielding.  In  addition,  the  circumstances  of  distance  learning 
require  robust  methodologies  for  instruction,  interaction  and  evaluation.  Even 
with  this,  the  opportunity  exists  for  major  advances  which  leverage  the  benefits 
flowing  from  the  people  and  the  developments  which  are  driving  the  ongoing 
information  revolution. 

Terms  of  Reference. 

a.  Review  the  Army  Distance  Learning  Plan.  Comment  and  make 
recommendation  on  priority  items  to  augment  the  current  program. 

b.  Recommend  training  development  and  delivery  in  the  post  -2010  era 
commonly  referred  to  as  “Army  After  Next".  Comment  on  the  concept  of  “Soldier 
Driven"  training  as  the  basis  for  training  management  in  AAN.  Discuss  its 
desirability  and  feasibility.  Comment  on  the  forms  it  might  take,  and  what  will  be 
required  of  soldiers  and  their  leaders  within  this  context. 

c.  Survey  business,  industry,  academia  and  government  for  innovative 
training  and  education  initiatives.  Ascertain  associated  technological, 
pedagogical,  psychological  or  other  interdisciplinary  approaches  aimed  at 
imparting  skills,  knowledge  and  ability  in  new  and  effective  ways. 

Study  Support.  Co-Sponsors  of  this  study  will  be  Ms.  Helen  T.  McCoy 
Assistant  Secretary  of  the  Army  (Financial  Management  and  Comptroller);  LTG 
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Leonard  D.  Holder,  Deputy  Commanding  General  for  Combined  Arms,  TRADOC; 
MG  J.M.  Riggs,  Assistant  Deputy  Chief  of  Staff  for  Operations,  and  Ms.  Sara 
Lister,  Assistant  Secretary  of  the  Army  (Manpower  and  Reserve  Affairs).  The 
Primary  Staff  Assistant  is  COL  Francis  Coppola,  Assistant  Chief  of  Staff  USAR 
Combined  Arms  Center.  Secondary  Staff  Assistant  is  Ms.  Paula  Rebar 
(ASA(FM&C)). 

Schedule.  The  study  panel  will  initiate  the  study  immediately  and 
conclude  its  effort  at  the  eleven-day  report  writing  session  on  June  16-26, 1997 
at  the  Beckman  Center  in  Irvine,  California.  As  a  first  step,  the  Study  Chair 
should  prepare  a  Study  Plan  for  presentation  to  the  Sponsor  that  outlines  the 
study  approach  and  study  schedule. 

Special  Provisions.  It  is  not  anticipated  that  this  inquiry  will  go  into  any 
"particular  matters"  within  the  meaning  of  Section  208,  Title  18  of  the  United 
States  Code. 


Sincerely, 


Assistant  Secretary  of  the  Army 
(Research,  Development  and  Acquisition) 
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PARTICIPANTS  LIST 


ARMY  SCIENCE  BOARD  S 
SUMMER  STUDY  ON 
“DISTANCE  LEARNING’’ 

Co-Chairs 


Dr.  J.V.  Braddock  LTG  (USA,  Ret.)  W.  Shoffner 

Founder 

BDM  International,  Inc. 


ASB  Panel  Members 

Ms.  Susan  G.  Lowenstam 

Attorney 

Dr.  L.  Warren  Morrison 

Visiting  Scientist 
Software  Engineering  Institute 
Carnegie  Mellon  University 


Dr.  Sue  E.  Berryman 

Senior  Education  Specialist 
The  World  Bank 

Dr.  Philip  C.  Dickinson 

Consultant 


Dr.  Edward  T.  Gerry 

Executive  Vice  President  and  CTO 
Schafer  Corporation 


Dr.  Richard  L.  Hartman 

President 

Advanced  Optical  Systems,  Inc. 

Dr.  Walter  B.  Laberge 

Associate  Director 

Institute  for  Advanced  Technology 

The  University  of  Texas  at  Austin 

Dr.  Peter  Lee 

Associate  Professor  of 
Computer  Science 
Carnegie  Mellon  University 

Dr.  Sung  M.  Lee 

Vice  Provost  for  Research  and 
Dean  of  the  Graduate  School 
Michigan  Technological  University 


Dr.  Harry  F.  O'Neil,  Jr. 

Professor  of  Education 
Psychology  and  Technology 
School  of  Education 
University  of  Southern  California 

Dr.  Karen  D.  Pettigrew 

Research  Mathematical  Statistician 
National  Institute  of  Mental  Health 
National  Institute  of  Health 

Dr.  Wilson  K.  Talley 

President 

Fannie  and  John  Hertz  Foundation 


Dr.  Harry  L.  Tredennick 

Consultant 


LTG  (Ret.)  Ronald  L.  Watts 

President 

Leadership  Development  Services 
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Staff  Assistant 


COL  Francis  Coppola 

Assistant  Chief  of  Staff 
USAR  Combined  Arms  Center 


Sponsors 


LTG  Leonard  D.  Holder,  Jr. 

Deputy  Commanding  General  for  Combined  Arms 
TRADOC 

Helen  T.  McCoy 

Assistant  Secretary  of  the  Army 
(Financial  Management  and  Comptroller) 

LTG  John  E.  Miller 

Deputy  Commanding  General 
TRADOC 


Gov't  Advisor 


MG  Johnny  M.  Riggs 

Assistant  Deputy  Chief  of  Staff 
DCS  OPS 


Cognizant  Deputy 


Herbert  K.  Fallin,  Jr. 

Director  for  Assessment  &  Evaluation 
SARDA 
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ACRONYM  LIST 


AAE 

Army  Acquisition  Executive 

AAN 

Army  After  Next 

AAR 

After  Action  Review/Report 

ABCS 

Army  Battle  Command  System 

AC 

Active  Component 

ADLP 

Army  Distance  Learning  Plan 

ADTDL 

Army  Doctrine  and  Training  Digital  Library 

ATT 

Advanced  Individual  Training 

ALSP 

Aggregate  Level  Simulations  Protocol 

ASA 

Assistant  Secretary  of  the  Army 

ASB 

Army  Science  Board 

AST&D 

American  Society  for  Training  &  Development 

ATRRS 

Army  Training  Resource  Requirements  System 

AVCSA 

Assistant  Vice  Chief  of  Staff  of  the  Army 

BCT 

Basic  Combat  Training 

CAC 

Combined  Arms  Center 

CACNET 

Combined  Arms  Center  Network 

CALL 

Center  for  Army  Lessons  Learned 

CAN 

Campus  Area  Network 

CAS  3 

Combined  Arms  and  Services  Staff  School 

CBI 

Computer  Based  Instruction 

CD 

Compact  Disk 

CD-ROM 

Compact  Disk  -  Read  Only  Memory 

CFO 

Chief  Financial  Officer 

CG 

Commanding  General 

CG  TRADOC 

Commanding  General  TRADOC 

CGSC 

Command  and  General  Staff  College 

CGSOC 

Command  and  General  Staff  Officer  Course 

CMU 

Carnegie  Mellon  University 

CONUS 

Continental  United  States 

COTS 

Commercial  off-the-shelf 

CSU-H 

California  State  University-Hayward 

CTC 

Combat  Training  Center 

CVW 

Collaborative  Virtual  World 

CW 

Courseware 

DA 

Department  of  the  Army 

DA-DCSOPS 

Department  of  the  Army  Deputy  Chief  of  Staff,  Operations 

DA-level  review  NLT 

Department  of  the  Army-level  review  no  later  than 
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DCG  CA  TRADOC 

Deputy  Commanding  General  Combined  Arms,  Training  & 
Doctrine  Command 

DCG  TRADOC 

Deputy  Commanding  General,  Training  &  Doctrine  Command 

DISC4 

Director  of  Information  Systems  for  Command,  Control, 
Communications  &  Computers 

DL 

Distance  Learning 

DoD 

Department  of  Defense 

FMC 

Financial  Management  &  Comptroller 

Force  XXI 

Force  21 

FORSCOM 

Forces  Command 

GOSC 

General  Officer  Steering  Committee 

GSU  DLPS 

Georgia  State  University  Distance  Learning  Programs  and 
Services 

HQS 

Headquarters 

ICT 

Integrated  Concept  Team 

IEEE 

Institute  of  Electrical  and  Electronics  Engineers 

ILE 

Installation,  Logistics,  Environment 

IPT 

Integrated  Product  Team 

IRR 

Individual  Ready  Reserve 

rrs 

Intelligent  Tutoring  System 

jrr 

Just-in-time 

KU 

Kansas  University 

LAN 

Local  Area  Network 

LM 

Lockheed-Martin 

MILSATCOM 

Military  Satellite  Communications  System 

MOSQ 

Military  Occupational  Specialty  Qualification 

MRA 

Manpower  and  Reserve  Affairs 

MSD 

Managing  Software  Development 

MY 

Man-Year 

NASA 

National  Aeronautics  &  Space  Administration 

NG 

National  Guard 

NG  PGM 

National  Guard  Program  Manager 

NGB 

National  Guard  Bureau 

NSC 

National  Simulation  Center 

NTU 

National  Technological  University 

O&M 

Organization  &  Maintenance 
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OA 

Organization  Assessment 

OC 

Observer  Controller 

OCAR 

Office  of  the  Chief  of  Army  Reserves 

OCONUS 

Outside  of  Continental  United  States 

OPFOR 

Opposing  Force 

OSU 

Ohio  State  University 

PC 

Personal  Computer 

PCC 

Pre-Command  Course 

PCS 

Permanent  Change  of  Station 

PEO 

Program  Executive  Officer 

PEORCAS 

Program  Executive  Officer  Reserve  Component  Automation 
System 

PEO  STAMIS 

Program  Executive  Officer  Standard  Army  Management 
Information  System 

Quarterly  IPR 

Quarterly  In  Progress  Review 

R&D 

Research  and  Development 

RAM 

Random  Access  Memory 

RBOC 

Regional  Bell  Operating  Centers 

RC 

Reserve  Component 

RDA 

Research  Development  &  Acquisition 

RL 

Residential  Learning 

ROM 

Read-Only  Memory 

SAT 

Systems  Approach  to  Training 

SATCOM 

Satellite  Communications 

SATS 

Standard  Army  Training  System 

SEI 

Software  Engineering  Institute 

SIMNET 

Simulations  Network 

SME 

Subject  Matter  Expert 

ST 

Site  Contractor 

TADLP 

Total  Army  Distance  Learning  Program 

TASS 

Total  Army  School  System 

TATS-C 

Total  Army  Training  System  -  Courseware 

TDY 

Temporary  Duty 

TNET 

Teletraining  Network 

TOR 

Terms  of  Reference 

TRADOC 

Training  and  Doctrine  Command 

TSP 

Training  Support  Packages 

TTHS 

Trainees,  Transients,  Holdees  and  Students 

UAH 

University  of  Alabama,  Huntsville 
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UMASS 

UNL 

USAR 

USDLA 

VCSA 

VTC 

VTT 


University  of  Massachusetts 

University  of  Nebraska,  Lincoln 

United  States  Army  Reserve 

United  States  Distance  Learning  Association 

Vice  Chief  of  Staff  of  the  Army 
Video  Teleconference 
Video  TeleTraining 


WAN 


Wide  Area  Network 
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SAMPLING  OVERVIEW 


The  following  brief  overview  and  accompanying  table  represent  the  compendium  of  information  collected 
from  the  various  sampling  visits,  meetings,  telephone  calls,  and  internet  searches  conducted  by  the  Panel 
Members.  This  overview  highlights  the  key  points  in  the  three  environments  (Industry,  Academia, 
Government)  explored. 

Industry 

-Similarity  of  challenge  and  operational  situation  with  Army 
After  preliminary  education  necessary  for  entry  level  position,  both  require 
continuing  education  with  minimal  interruption  of  ongoing  operations 

-  Industry  has  used  distance  learning: 

To  achieve  necessary  education  at  minimum  cost  and  loss  of  operational  efficiency 
effectively  in  augmenting  skills  and  saving  travel  time  and  costs 

-  Industry,  largely  classroom  based,  is  tending  to  move  from  a  small  number  of  people  trained  through 
simultaneous  remote  classroom  education  to  reliance  on  training  using  the  internet  for  JIT  information 

-  Industry  uses  wide-bandwidth  DL  when  incident  to  already  existing  administrative  data  channels 

Academia 

-  Undergraduate  degrees  from  established  universities  predominantly  granted  via 
residential  training 

-  Virtual  universities  also  granting  nonresidential  undergraduate  degrees 
Advanced  degree  education  increasingly  nonresidential 

-  User  needs  have  driven  universities  to  provide  nonresidential  learning 

-  Non  residential  education  found  to  be  equal  to  residential 

-  Studies  show  that  DL  fosters  greater  student  interaction;  Greater  student  interaction  improves 
education 

-  Decreased  emphasis  on  multi-media  techniques  occurring: 

Lower  band-width  asynchronous  techniques  foster  greater  student  interaction 
A  large  amount  of  material  is  readily  available  on  Internet 

Government 

-  The  DoD  has  been  studying  DL  on  an  on-going  basis;  It  has  been  slow  to  implement  DL  because 
savings  are  difficult  to  quantify 

-  The  most  innovative  examples  to  date  have  been  in  the  National  Guard  and  Reserves 

-  DL  is  often  tied  in  with  state  initiatives 

-  Currently  available  courseware  is  used  without  the  long  delays  inherent  in  courseware  conversion 

-  DoD  dependent  school  system  has  also  been  a  pioneer  in  DL 
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INSTITUTE  FOR  DEFENSE  ANALYSIS 


Example-Studies 

•  Sample  -  Institute  for  Defense  Analyses  (IDA).  IDA  is  a  Federally  Funded  Research  and  Development  Center  that  has  a 
primary  function  of  conducting  studies  and  analyses  for  the  Office  of  the  Secretary  of  Defense.  One  of  its  areas  of  interest  is 
the  cost  and  effectiveness  of  DoD  education  and  training.  The  IDA  has  written  well-researched  reports  concerning  computer- 
based  instruction  courseware  and  DL.  Its  reports  also  survey  industrial  and  university  applications.  IDA  Paper  P  -  2567 
Interactive  Courseware  (ICW)  and  the  Cost  of  Individual  Training,  November  1992.  Training  Challenges  -  Many 
commentators  have  discussed  current  trends  that  increase  the  challenges  to  successful  conduct  of  military  training.  It  is 
unlikely  that  conventional  approaches  using  platform  lecture,  paper-based  workbook  exercises  and  laboratory  experience  with 
actual  scarce  and  expensive  equipment  will  meet  the  demands  for  the  training  efficiency  and  effectiveness  that  are  necessary 
for  the  coming  century.  Increasingly,  trainers  in  both  military  and  civilian  settings  are  turning  to  technology  as  a  source  of 
improved  training  effectiveness  and  efficiency. 

•  Technology  Costs  and  Effectiveness  -  Technology  can  be  used  to  teach.  It  improves  instructional  effectiveness  and  reduces 
time  to  reach  instructional  objectives.  Students  enjoy  using  technology.  Technology  also  lowers  instructional  costs  and 
increases  readiness.  This  sampling  of  results  is  neither  comprehensive  nor  conclusive,  but  it  strongly  suggests  that  applications 
of  technology  in  military  training  may  be  more  effective  and  less  costly  than  current  practices.  Therefore,  it  is  reasonable  to 
argue  that  the  resources  needed  for  initial  investment  in  these  approaches  may  be  well  spent.  These  resources  will  include 
funding,  time,  and  effort  to  affect  significant  changes  in  professional  practice  and  instructional  institutions. . 

•  Budgetary  Considerations  -  Traditionally,  budget  decisions  have  tended  to  focus  almost  exclusively  on  the  potential  for 
savings  within  the  operations  and  maintenance  (O&M)  accounts.  There  are  O&M  savings  to  be  gained  from  investments  in 
converting  from  current  training  approaches  to  those  that  are  technology  based,  but  there  are  significant  additional  payoffs  to 
consider  as  well.  These  accrue  from  the  reductions  in  student  time  needed  to  train  when  technology  is  used  and  the 
concomitant  increase  in  time  these  individuals  are  available  for  duty.  One  difficulty  is  that  although  the  investment  needed  to 
convert  training  programs  will  most  probably  come  from  O&M  accounts,  major  savings  will  appear  in  personnel  accounts,  not 
in  O&M.  It  is  necessary  to  quantify  the  potential  payoff  in  terms  of  student  pay  as  well  as  the  O&M  savings  generated  from 
investments  in  training  technology. 

Contactinformatioiu  Dr.  Dexter. Fletcher.  703-578-2837  “fletcher@ida.org” _ _ 

CALIFORNIA  STATE  UNIVERSITY,  HAYWARD  (CSU-H) 

Example-Networked  Students  and  Instructors 

•  Sample  -  CSU-H  “The  transmission  medium  isn’t  as  important  as  the  amount  of  preparation  of  the  material  to  be  taught.”  Dr. 
Roger  Parker,  Director,  Instructional  Media  Center/Distance  Learning,  CSU-Hay  ward 

•  Observations  -  The  22  campuses  of  CSU  are  interconnected  with  a  variety  of  point-to-point-microwave,  ISDN,  fiber  optic 
cable,  coax,  and  standard  telephone  lines.  CSU  is  now  exploring  C  and  Ku  band  satellites  using  3  meter  and  3.7  meter  dishes. 

The  following  definitions  were  used:  Synchronous  learning  requires  the  teacher  and  the  student(s)  to  be  somewhere  at  a 
certain  time.  The  teacher  and  the  class  may  be  physically  remote,  but  the  two  are  simultaneously  in  communication. 

Asynchronous  learning  allows  the  student  to  choose  the  speed,  time  and/or  place  of  instruction. 

DL  has  been  in  existence  since  the  first  correspondence  course,  which  is  a  good  example  of  asynchronous  learning.  Contrast 
this  with  the  idea  that  a  school  consists  of  a  "log"  with  a  teacher  sitting  on  one  end  and  a  student  on  the  other-  synchronous 
learning.  The  CSU  System  is  using  DL  for  teaching  both  cognitive  and  motor  skills.  CSU  has  participated  in  various  studies 
to  see  how  DL  compares  with  traditional  and  other  learning  modes.  The  result  is  that  the  transmission  medium  is  not  as 
important  as  the  amount  of  preparation  of  the  material  taught.  A  course  that  must  stand  completely  alone,  asynchronous,  takes 
an  incredible  amount  of  work  to  prepare.  It  is  only  practical  for  organizations,  such  as  General  Motors  and  the  Army,  which 
have  to  teach  the  same  material  repeatedly  to  very  large  numbers  of  individuals. 

Contact  information;  Dr.  Roger  Parker •.  Director .  Instructional  Media  Center/Distance  Learning.  510-885-3692  510-885-3953 

(Fax}  rparker@csuhavward.edu _ _ _ _ _ _ 
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CMU  INTELLIGENT  TUTORING  PROJECT 

Example  -  Networked  Classrooms 

•  Sample  -  CMU  Intelligent  Tutoring  Project 

•  Purpose:  Apply  cognitive  psychology  and  established  teaching  principles  in  design  of  artificially  intelligent  tutoring  systems. 

•  Infrastructure:  Practical  Algebra  Tutor  (“PAT”)  is  in  use  in  four  high  schools  nationally.  Classrooms  equipped  with  network  of 
workstations,  along  with  teacher. 

•  Observations:  The  CMU  Intelligent  Tutoring  Project  is  a  research  project  investigating  the  use  of  artificial  intelligence 
techniques  in  a  computer-based  tutoring  system.  The  project  members  include  cognitive  psychologists  and  computer  scientists 
working  together  to  apply  established  teaching  principles  in  the  design  of  tutoring  systems.  The  main  demonstration  system 
produced  by  the  project  is  the  Practical  Algebra  tutor,  which  teaches  high-school  algebra.  Currently  deployed  in  four  high 
schools,  PAT  has  shown  itself  highly  effective  in  teaching  concepts  in  algebra.  The  main  emphasis  of  the  system  is  to  stress 
cognitive  development  by  forcing  students  to  think  abstractly.  It  does  this  by  posing  problems  and  then  structuring  the 
student’s  interaction  with  the  problem  so  it  can  be  viewed  in  several  different  ways,  such  as  by  equations,  words,  graphs,  and 
pictures.  The  PAT  makes  extremely  aggressive  use  of  computer  technology  to  maximize  transfer  of  cognitive  and  conceptual 
skills.  Performance  of  students  in  standardized  tests  shows  clear  superiority  over  traditional  classroom  learning.  However, 
construction  of  course  material  is  extremely  difficult  and  relies  on  expert  support.  Currently,  CMU  IT  is  a  research  concept, 
not  a  product. 

Contact  Information:  DR  Ken  Koedineer ,  Human-Computer  Interaction  Institute  ♦  Carneeie  Mellon  University ,  412-268-7667 

“koedineer@cmu.  edu ” _ _ _ 

CMU  JUST-IN-TIME  LEARNING 

Example  -  Non-Networked  Students 

•  Sample  -  CMU  Just-In-Time-Lectures  (JITL) 

•  Purpose:  Exploit  computer  technology,  but  with  reduced  delivery  and  productions  costs. 

•  Infrastructure:  PC/Mac  and  CD-ROM  based.  (Special  “hyper-lecture”  software  for  multimedia  presentation  of  learning 
material) 

•  Metrics:  Goal  is  less  than  10  hours  of  production  time  per  hour  of  DL  product.  (Simple,  inexpensive,  and  widely  available 
production  equipment) 

•  Observations:  Simple  video,  either  non-networked  via  videotape  or  networked  via  VTC,  has  the  advantage  of  being  relatively 
inexpensive  to  produce  and  deliver.  In  addition,  simple  video  is  a  delivery  mode  that  is  familiar  to  both  teachers  and  students. 
However,  video  also  has  a  number  of  limitations.  Videotape  is  non-interactive  and  linear.  This  makes  it  hard  for  the  teacher  or 
student  to  ask  questions,  and  little  or  no  support  for  topic  search  is  available.  The  CMU  Just-In-Time-Learning  (JITL)  concept 
is  to  exploit  computer  technology  to  enhance  video-based  DL  while  retaining  the  advantages  of  relatively  inexpensive 
production  and  simple  delivery.  A  JITL  comes  on  a  CD-ROM  and  contains  software  that  will  run  on  any  PC  or  Macintosh 
system.  When  played,  the  CD  shows  a  video,  usually  containing  a  lecturer  super-imposed  on  a  second  video  track  of 
synchronized  multimedia  material.  The  multimedia  material  can  be  simple  PowerPoint  presentation  slides,  computer-based 
graphics/animation,  or  video  demonstration  of  a  skill/concept.  A  second  window  shows  a  scrolling  outline  of  the  lecture.  The 
student  can  click  on  this  for  options  on  replay,  skipping  forward  or  back,  and  searching  for  specific  topics  of  interest.  Finally, 
there  are  a  number  of  buttons  for  assessment  tests,  asking  and/or  answering  questions,  and  access  to  on-line  supplementary 
materials. 

Contact  Information:  DR  Roeer  Dannenbers.  Associate  Professor ,  School  of  Computer  Science ,  412-268-3827 

“roeer.dannenserz@cs.cmu.edu” 
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Example  -  Networked  Students 
•  Sample  -  CMU  Online 


CMU  ON  LINE 


•  Purpose:  Provide  self-paced  and  interactive  learning  for  college  students. 

•  Infrastructure:  PC/Mac,  Internet,  and  written  materials. 

(Online  material  and  assessment  tests  are  tracked  and  generated  uniquely  for  each  student,  according  to  his/her  observed 
competence.  Utilization:  Over  300  students  learning  individually,  for  college  credit.) 

•  Observations:  The  CMU  Online  system  uses  computer  technology  to  tailor  course  content  and  assessment  to  individual 
student  strengths  and  weaknesses.  The  course  materials  and  assessment  exercises  are  created  and  stored  in  a  database  system. 
The  DL  system  is  then  made  available  through  a  World  Wide  Web  interface  (Netscape).  The  course  topics  are  organized  in  a 
“non-linear”  fashion  that  allows  students  to  find  specific  topics  of  interest  and  choose  the  order  of  presentation.  As  students 
complete  the  course  topics,  the  DL  system  generates  tests  and  homework  assignments  automatically,  taking  into  account  the 
student’s  performance  on  previous  tests  and  assignments.  In  many  cases,  the  software  system  also  does  grading  and  feedback 
diagnosis  automatically,  though  in  some  cases,  a  human  instructor  is  required  to  do  full  assessment  and  grading.  Performance 
on  tests  shows  that  the  DL  students  do  at  least  as  well  as  RL  students.  No  time  pressures  are  placed  on  the  DL  students.  This 
possibly  accounts  for  the  good  performance;  however,  it  also  means  that  many  students,  about  one-third,  fail  to  complete  the 
full  course.  Organization  and  creation  of  course  content  suitable  for  software-customized  delivery  and  assessment  is  laborious. 

Contact  Information:  DR  Peter  Lee ,  School  of  Computer  Science ,  CMU,  412-268-6000 ,  peter.lee@cs.cmu.edu _ 

CMU  SEI  MANAGING  SOFTWARE  DEPARTMENT 
(MSD)  COURSE 

Example  -  Non-Networked  Students 

•  Sample  -  SEI  “Managing  Software  Development”  (MSD)  course. 

•  Purpose:  Allow  non-resident  students  to  complete  portion  of  MS  requirement  at  SEI. 

•  Infrastructure:  Small  modules  (2  weeks)  with  videotaped  lectures.  (Tutor-presented  video.)  Live  chat  sessions  via  Java-based 
system  on  WWW.  Email  with  instructors. 

•  Observation:  Carnegie  Mellon’s  Software  Engineering  Institute  (SEI)  offers  a  Master  of  Software  Engineering  (MSE)  degree. 
Some  courses  for  the  MSE  are  now  offered  via  DL.  Live  lectures  via  VTC  were  previously  used.  Later,  these  were  dropped  in 
favor  of  videotape  when  it  was  learned  that  more  than  80%  of  the  recipients  were  doing  their  own  videotaping  anyway. 
Currently,  a  live  tutor  presents  the  videotapes.  The  tutor  is  able  to  stop  the  tape  to  ask/answer  questions  as  well  as  replay 
segments  as  students  request  them.  In  order  to  ensure  that  each  student  has  enough  time  pressure  to  complete  the  course,  each 
module  is  kept  short,  roughly  2  weeks  each.  Each  module  is  supported  by  written  and  WWW  materials  and  is  terminated  with 
an  assessment  test  as  well  as  a  live  chat  session  on  the  WWW.  The  approach  has  proven  to  be  cost-effective  as  well  as 
effective  in  teaching.  DL  students  perform  at  least  as  well  as  the  RL  students. 

Contact  Information:  Dr.  Jim  Tomayko ,  Professor ,  CMU ,  MSE  jet@cs.cmu.edu _ 

UNIVERSITY  OF  ALABAMA,  HUNTSVILLE  (UAH) 

Example-Networked  Classrooms 

•  Sample:  Georgia  Tech,  U.  of  Alabama  in  Huntsville,  and  NTU 

•  Observations:  This  information  comes  from  research  papers  and  an  interview  with  the  professor  concerned.  The  UAH 
professor  was  invited  to  teach  a  graduate  level  NTU  course  in  management.  The  nearest  source  location  was  Georgia  Tech,  a 
four-hour  drive.  The  professor  decided  to  try  a  somewhat  controlled  experiment  in  DL.  The  student  body  consisted  of  a  live 
class  at  Georgia  Tech  and  NTU  students  spread  throughout  the  country  who  simultaneously  viewed  a  video  telecast.  The  NTU 
students  had  an  audio  link  back.  The  professor  taught  the  same  class  at  UAH  to  a  live  class,  on  a  different  day  of  the  week. 

The  professor  met  twice  with  the  NTU  students  at  group  meetings.  All  students  took  the  same  tests.  All  tests  were  scored  in  a 
blind  manner  so  that  the  " scorer”  did  not  know  which  class  the  student  was  taking.  The  NTU  DL  students  scored  significantly 
better  than  the  live  class  students  did.  Analysis  and  observation  provided  the  following  insight.  The  NTU  students  were  active 
employees,  and  each  felt  that  completing  NTU  courses  and  a  degree  would  enhance  their  career  with  their  current  company. 


UAH  participants  were  part-time  students  who  mostly  looked  on  the  degree  as  a  way  to  get  a  better  job  with  a  new  employer. 

The  Georgia  Tech  students  were  full-time  students  taking  a  course  to  fill  requirements.  The  hypothesis  is  that  NTU  students 
were  interested  in  knowledge,  the  UAH  students  in  degrees,  and  the  Georgia  Tech  students  in  credit  hours.  In  the  tests,  the 
NTU  students  scored  best  on  practical  problem  solving  and  in-group  problems.  The  Georgia  Tech  students  scored  best  on 
purely  theoretical  problems.  The  UAH  students  scored  the  worst  on  group  collaboration.  Both  the  Georgia  Tech  and  UAH 
live  classroom  students,  by  questionnaire,  had  little  interaction  with  each  other  outside  the  classroom  or  interaction  with  the 
professor.  In  contrast,  the  NTU  students,  who  had  to  communicate  via  email,  developed  an  intense  interaction.  Problems  were 
solved  jointly.  In  fact,  collaboration  went  beyond  the  class.  Students  helped  each  other  with  company  problems  and  developed 
business  relations  between  their  companies.  The  students  also  interacted  much  more  intensively  with  the  professor,  sending  in 
problem  solutions  one  at  a  time  and  getting  feedback  before  the  next  problem.  Observation  of  the  NTU  classroom  showed  that 
the  students  were  looking  at  the  pre-delivered  notes  and  a  textbook  while  listening  to  the  professor,  rarely  was  the  screen 
viewed.  In  contrast,  the  in-class  students  only  looked  at  the  professor  and  were  much  less  actively  involved.  The  professor 
concluded,  in  addition  to  these  observations,  that  training  the  teacher  is  critical,  and  instructing  this  course  took  significantly 
more  work  than  a  conventional  course  because  of  the  intensive  student  interaction. 

GEORGIA  STATE  UNIVERSITY 

Example-Networked  Classrooms,  Students  and  Instructors 

•  Sample:  The  Department  of  DL  Programs  and  Services  at  Georgia  State  University 

•  GSAMS  -  The  Georgia  Statewide  Academic  and  Medical  System  (GSAMS)  -  videoconferencing  network  providing  fully 
interactive,  two-way  DL  environment. 

•  Interactive  Technology  Project  -  State's  proposal  to  improve  existing  distance  learning  environments,  project  status,  pilot 
projects,  etc. 

•  Instructional  Television  Fixed  Service  -  ITFS  (Instructional  Television  Fixed  Service)  -  channels  providing 
educational/informational  programming  carried  by  a  cable  tv  company.  Educational  programming  from  numerous  sources 
to  be  delivered  on  ITFS  full  implementation,  Fall  1998.  GSU’s  Division  of  Distance  Learning  Programs  and  Services 
coordinates  ITFS  consortium  programming  schedule;  system  testing  and  phase-in  of  educational  offerings,  Fall  1997. 

•  Special  Programs  -  World  Food  Day  Teleconference,  the  Picasso  Program  with  the  High  Museum,  etc.;  GSU  DLPS 
Business  Partnerships  -  direct  and  indirect  include:  Bell  South  Cable,  Apex  Museum,  High  Museum,  Zoo  Atlanta, 
Fembank,  Adult  Literacy  Program,  Georgia  Tech,  University  of  Georgia,  Dekalb  Community  College,  Georgia  Public 
Television,  Kennesaw  State  University,  Michael  J.  Coles  School  of  Business,  ACTV,  CNN,  Division  of  Continuing 
Education  at  GSU,  World  Food  Day,  Gerontology  Program  (involves  8  Georgia  universities) 

•  TREES  Program  -  A  special  technology  utilization  co-reform  program  with  funding  from  the  Coca-Cola  Foundation 
called  Developing  Technology  Rich  Educational  Environments.  Program  goals  are  enhancing  educators  ability  to  use 
technology  in  their  work  and  collaborating  technology  use  more  effectively  with  Georgia’s  K-12  sector  and  the 
university  system.  Program  initiated  in  1995;  tests  indicate  it  is  effective  at  introducing  faculty  at  all  levels  to  the 
technology  use  in  education.  Sessions  in  this  program  teach  the  educators  on  being  DL  instructors;  the  creation  and 
use  of  multi-media,  integrating  multi-media  into  new  and  existing  courses,  integrating  technology  into  the  classroom 
and  selecting  and  preparing  their  own  CD  educational  products. 

•  Gerontology  Collaboration  -  Provides  information  on  the  Distance  Learning  partnership  in  gerontology  through  a 
description  of  the  initiative,  list  of  the  institutes  involved,  and  the  purpose  of  the  project. 

•  Observations:  DL  is  rapidly  maturing  as  a  highly  effective  method  to  deliver  education  and  information  to  a  dispersed 
audience.  The  Georgia  Department  of  Education  has  installed  over  300  DL  sites  in  the  state,  located  in  institutions  of  higher 
education,  selected  medical  institutions,  nearly  all  elementary  schools,  and  most  high  schools.  A  major  portion  of  these  sites 
are  part  of  the  Georgia  Statewide  Academic  and  Medical  System  (GSAMS).  These  sites  serve  as  the  primary  backbone  of  the 
state's  delivery  of  DL  via  videoconferencing.  Extensive  Internet  World-Wide-Web  pages  provide  faculty,  educators,  support 
personnel,  and  students  with  guidelines  for  developing  and  delivering  DL  education  and  communication  in  a  typical  classroom 
setting.  Particularly  notable  is  the  Distance  Learning  Handbook  at 

http://www.gsu.edu/webprj01/adm/wwwdls/public_html/Presenters/NetIt24/7356x.htm 
Contact  information :  :  Georgia  State  University  Division  ofDistance  Learning  Programs  and  Services  j DLPS )  (404}  651- 177 
Carla  Relafordt  DLPS  Director .  retoford@gSM.edM;  GSU  DL  website :  http://www.ssu.edu/~wwwdls/ _ 
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KANSAS  UNIVERSITY 


Example-Networked  Classrooms  and  Students 

•  Sample:  Current  DL  applications  at  Kansas  University  (KU) 

•  Continuing  Education  with  VA 

•  Offering  DL  course  on  how  to  develop  DL  courses 

•  Experience  with  DL  in  Norway 

•  Civil  Engineering  courses  via  DL 

•  Observations: 

•  Asynchronous  teaching  to  meet  the  needs  of  the  students 

Classes  ’'broadcasts"  three  times  a  week 
Students  work  at  their  convenience 
Respond  to  homework  via  internet 

•  Problem  oriented  study  -  building  a  bigger  bridge  challenges  students 

•  Follows  model  used  by  architecture  program 

•  Student  collaboration  via  the  Internet  provides  strong  reinforcement  of  training;  Students  tend  to  solve  problem  among 
themselves 

•  Those  unanswered  questions  are  structured  and  fed  to  instructor  via  Internet 

Contact  Information:  Sandra  W.  Gautt  (swz@co.wvo.ukans.edu) _ 

LOCKHEED-MARTIN  MISSILES  &  SPACE 

Example-Networked  Classroom 

•  Sample  -  Lockheed  Missiles  and  Space,  Sunnyvale,  CA 

•  Observations:  All  computer  operations  of  the  firm  are  centralized.  CBI  is  the  normal  mode  for  training  and  instruction  within 
that  group;  the  group  is  comprised  of  some  five  thousand  people.  Courses  are  taught  in  sixty  to  ninety  minute  sessions  to  four 
or  five  hundred  people  each  week.  Courses  are  offered  not  only  in  applications  and  language  training,  but  also  security, 
environmental  training  and  health,  and  diversity  training.  The  main  reason  for  using  CBI  is  savings,  followed  by  uniformity  of 
training.  There  appears  to  have  been  no  analysis  as  to  the  use  of  CBI  versus  synchronous  DL  or  traditional  teaching  methods. 
However,  the  trend  at  LM  appears  to  be  toward  movement  onto  the  Internet  rather  than  diskettes,  cassettes,  or  CD’s.  This  is 
partly  due  to  the  need  to  adapt  to  regional  variations. 

Contact  Information:  Pat  Anderson  Executive  VP,  LM  Enterprise  Information  Systems _ 

NASA  AMES  RESEARCH  CENTER 

Example-Networked  Classrooms  &  Students 

•  Sample  -  NASA  AMES  Research  Center 

•  Observations:  Web-based  K-12+  multimedia  DL  through  Virtual  Conferencing  (Lectures,  participants,  chat  windows,  video 
capability,  300  user  maximum);  university  courses  (lectures,  slides,  transcripts,  live — virtual  conferencing — or  recorded);  live 
on-line  (Antarctica,  Jupiter,  shuttle,  Hubble,  Mars,  Mount  Wilson  remote  control).  NASA  Ames  has  set  up  a  DL  program  to 
service  local  K-12  schools.  Despite  a  heavy  investment  in  equipment,  the  lack  of  resources  allocated  for  the  content  of  the 
lessons  were  apparent.  There  were  also  technical  problems  with  the  LAN  that  was  used  to  demonstrate  some  of  the  classes. 
NASA  might  be  expected  to  show  live  pictures  from  space  to  interest  students,  but  the  material  which  the  Panel  saw  was  Home 
Page-like  with  canned  stills  and  video  clips.  The  elements  of  a  successful  program  are  there,  but  not  the  program  itself. 

Contact  Information:  Mark  Leon ,  NASA  Ames ,  Moffett  Field ,  CA _ _ 

ONE  TOUCH  SYSTEMS 

Example-Networked  Classrooms 

•  Sample:  One  Touch  Systems,  San  Jose,  CA 

•  Observations:  One  Touch  is  in  the  satellite  teaching  business  with  high  band  transmission  and  low  band  response  from  the 
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students.  By  so  doing,  live  classroom  experience  of  satellite-based  instruction  can  occur.  In  a  synchronous  mode,  the 
instructor’s  image  is  beamed  to  one  or  more  classrooms  via  satellite  and  down  link.  At  points  during  the  lecture,  the  instructor 
displays  a  question  with  multiple  choice  answers.  Each  student  has  a  keypad  that  is  signed  on  at  the  start  of  class  using  their 
name  and  ID.  The  student's  individual  answers  are  recorded  by  pressing  a  key  on  the  pad.  All  responses  are  returned  to  the 
lecturer  via  land  lines  and  are  interpreted  by  the  Site  Controller  (ST),  a  computer.  After  an  appropriate  time,  the  ST  produces 
an  analysis  of  the  answers,  which  the  instructor  may  share  with  all  of  the  students.  The  instructor  will  be  able  to  explain  which 
answer  was  correct  and  which  incorrect  answer  was  most  often  chosen.  The  instructor  may  then  call  on  individual  students  for 
comments;  the  ST  has  that  data  available.  If  the  pace  of  instruction  is  too  fast,  the  student  may  press  a  "panic"  button.  A 
screen  presents  the  sum  of  all  such  inputs  to  the  instructor,  perhaps  as  a  rising  column  of  red.  The  teacher  may  simply  slow 
down  if  it  becomes  clear  that  the  pace  is  moving  too  quickly.  The  system  is  in  use  at  Ford  where  twenty-three  host  sites  with 
fifty  to  sixty  hours  per  day  on  over  10  channels  and  up  to  14,000  "classrooms"  are  used.  There  were  220,000  taught  in  1997. 
Government  agencies,  such  as  FAA,  are  using  the  system,  and  elements  within  the  DoD  and  the  Army  are  buying  into  it. 

Contact  Information:  Ken  Chow » Director  of  Marketing.  One  Touch  Systems ,  San  Jose .  CA _ 

RAND  CORPORATION 

Example-Research  &  Development 

•  Sample:  The  RAND  Corporation 

•  Observations:  The  RAND  Corporation  is  a  non-profit  federally  funded  research  and  development  corporation.  It  conducts 
military  and  non-military  research  and  policy  analysis.  This  R&D  effort  is  performed  for  the  U.S.  Department  of  Defense  and 
its  service  branches,  non-defense  agencies  of  federal  and  state  governments,  associations  such  as  the  California  Education 
Roundtable,  foundations,  cities,  and  foreign  clients,  including  the  European  Union.  Its  headquarters  is  in  Santa  Monica, 
California,  and  a  branch  office  is  located  in  Washington,  D.C.  and  representation  positioned  in  New  York  City  and  Delft,  the 
Netherlands. 

Its  Institute  on  Education  and  Training  publishes  on  issues  of  DL,  such  as  The  Intelligent  Tutor  for  Basic  Algebra,  Overview  of 
Object-Oriented  Microworlds  for  Learning  Mathematics ,  and  Untangling  the  Web :  Applications  of  the  Internet  and  Other 
Information  Technologies  to  Higher  Education . 

Contact  information:  Dr .  Roeer  Beniamin .  Director \  Institute  on  Education  and  Training  The  RAND  Corporation ,  1700  Main 

St ..  Santa  Monica ,  CA.  90407-2138;  (310)  393041  L  Ext.  6745;  Roser-Benjamin@RAND.ors . _ 

SUN  MICRO  SYSTEMS 

Example-Networked  Systems 

•  Sample:  Sun  Micro  Systems,  Palo  Alto,  CA 

•  Observations:  Because  Sun  makes  and  sells  workstations,  its  presentation  was  concerned  with  the  use  of  a  workstation  in  DL 
situations. 

Sun  has  established  LANs  with  PCs  that  are  able  to  download  exceptional  graphics  from  a  Sun  server. 

For  such  systems,  network  bandwidth  appears  to  be  the  constraint.  Within  the  LAN,  there  does  not  seem  to  be  such  a  problem. 
For  the  Army,  there  are  likely  to  be  solutions  to  problems  that  will  work  with  any  given  bandwidth.  Some  problems  are  likely 
to  require  more  bandwidth  than  is  currently  available. 

Contact  Information:  John  Dutra ,  Director ,  Network  Architecture ;  Anne  Schowe,  VP  and  GM ,  Interactive  Services  Group; 

David  Ballard,  Director \  Government  Marketing*  Sun  Micro  Systems,  Palo  Alto ,  CA _ 

CALIFORNIA  VIRTUAL  UNIVERSITY 

Example-Virtual  University 

•  Sample:  CALIFORNIA  VIRTUAL  UNIVERSITY: 

“Intellectual  property  rights  probably  are  a  bigger  problem  than  installation  of  even  more  physical  equipment.”  Dr.  James 
Burke 

•  Observations:  Dr.  Burke  is  a  consultant  to  the  California  Virtual  University  initiative.  It  is  California’s  response  to  the  13  State 
Western  Virtual  University.  The  CVU  is  minimally  funded  by  the  State  of  California.  The  players  include  (outside  the 
Governor's  Office)  the  University  of  California  System,  the  California  State  University  System,  several  private  universities, 
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and  some  federal  installations. 

There  has  been  little  attention  paid  to  the  intellectual  property  rights  inherent  in  a  system  that  would  allow  a  student  in,  say, 
Chico,  to  take  graduate  courses  from  teachers  at  Stanford,  UC-Irvine,  CSU-San  Diego,  and  other  places  and  get  the  graduate 
degree  from  the  University  of  Southern  California.  A  part  of  the  problem  is  that  TV  teaching  is  well  rewarded  at  Stanford  and 
carries  almost  negative  awards  at  UC  campuses. 

A  minor  problem  is  the  physical  equipment:  it  is  believed  that  by  the  time  the  course  content  and  ownership  rights  are  worked 
out,  the  K-12  classrooms  throughout  the  entire  state  will  be  available  to  serve  as  auxiliary  classrooms  for  the  CVU.  Hence,  no 
major  investment  in  equipment  and  facilities  will  be  needed  if  appropriate  agreements  are  worked  out. 

Contact  Information:  Dr.  James  Burke.  Consultant  Governor ]s  Office  on  the  CVU _ 

UMASS  DEPARTMENT  OF  COMPUTER  SCIENCE 

Example-Courseware  Development 

•  Sample  -  University  of  Massachusetts,  Department  of  Computer  Science 

•  Purpose — Focus  on  intelligent  systems  for  learning,  modeling,  control  issues  and  inference  mechanisms  for  human- 
machine  communications 

•  Infrastructure — Center  for  Knowledge,  Communication  and  Computer-based  Instructional  Technology 

•  Metrics— Episodic  memory  in  student  modeling  to  evaluate  student  performance  is  being  developed 

•  Courseware  development  — MEDTECH  Tudor  Anatomy  &  Physiology 

•  Observations 

•  Example  of  an  applicable  course  taught  self  paced  over  an  1 8  month  duration  cut  to  6  months 

•  Training  Development  team  composed  of  programmer,  researcher  &  SME 

•  Dual  Use  Approach  for  applications  benefiting  academia  &  military 

•  Student  modeling  metrics  development  working  at  a  baseline  level  with  continued  R&D  for  comprehensive  progression 

Contact  Information:  Beverly  Wool£4L3^545^4265_^bev@csMfnass^edu^ _ _ _ _ _ 

UNL  DEPARTMENT  OF  DISTANCE  EDUCATION 

Example-Courseware  Development 

•  Sample-University  of  Nebraska,  Lincoln,  Department  of  Distance  Education 

•  Purpose — Communications,  Learning,  and  Assessment  in  a  Student-centered  System  (CLASS)  for  electronic  interaction 
among  students,  instructors  and  external  resources 

•  Infrastructure — 22,000  students  in  more  than  135  countries 

•  Metrics — Course  Map  tool  charts  student’s  exploration  of  a  variety  of  paths  through  material  for  learning  activities  and 
experiences 

•  Courseware  development  —Global  Perspectives:  Bosnia,  Environmental  Chemistry,  Writing  and  Informal  Geometry 

•  Observations 

•  CLASS  courses  provide  the  following  benefits:  for  teachers  to  monitor  students  progress;  for  students  to  pace  their 
individual  progress;  for  courses  to  accommodate  various  learning  styles;  for  instructors  and  students  to  access  a  broad 
array  of  resources;  and  for  instructors  to  develop  levels  of  skill  and  comfort  with  technology; 

•  Course  Production  team  &  Curriculum  Development  Unit  for  course  sustainment 

•  Partnership  between  R&D,  Telecommunications  &  Education  Communities 

Contact  Information:  Charlotte  Hazzard  402-472-0884  “hazzard@unlinfo.unl. edu” _ _ _ 

OHIO  STATE  UNIVERSITY 


Example-Networked  Classrooms 

•  Sample  -  Ohio  State  University 

•  Purposes 

•  Enhanced  learning  environment  for  enrolled  students 

•  New  Revenue  Generator 

_ •  Student  and  Corporate _ 


D-10 


•  Outreach 

•  Statewide  to  K- 12 

•  Closer  relations  with  alumni 

•  How  is  it  done? 

•  CDs,  Interactive  video,  Video  delivered  to  desktop,  Web 

•  Metrics? 

•  No.  Only  rudimentary  evaluation.  Faculty  committee  with  student  representation 

•  Courseware 

•  Some  outsourcing 

•  Mostly  faculty  member  does  content  using  a  standardized  set  of  templates 

•  Shared  staff  to  assist  faculty 

•  Student  trainees  also  assist 

•  Best  practices  server  on  web 

•  Other 

•  Member  of  “Learn  Share” 

•  Consortium  of  9  Fortune  500  Cos.,  3  Universities  and  State  of  Ohio 

•  Develop  training  materials  and  research  pilots  on  effectiveness  of  different  technology  methods 

Contact  Information:  Jim  Davis ,  Associate  Provost  &  Director  of  University  Technical  Services  614-292-6553 

“jfdavis@osu.edu  ”  _ 

MITRE  CORPORATION 

Example-Courseware  Development 

•  Sample  -  MITRE  Corporation 

•  Purpose — Intelligent  Collaborative  Learning  Environments,  Intelligent  Tutoring  Systems  (ITS)  and  Computer-supported 
Collaborative  Learning  (CSC L)  tools. 

•  Infrastructure — R&D  Efforts 

•  Metrics — Learning  Companion  to  stimulate  effective  coaching 

•  Courseware  development  — Collaborative  Virtual  World  (CVW) 

•  Observations 

•  Enhancement  of  computer  supported  collaborative  work  environments  to  supply  life-long  learning  and  support  for  the 
work  force  for  more  efficient  and  effective  workflow. 

•  Simulated  peer  partners  with  the  student  to  more  clearly  lay  out  reasoning  so  ITS  will  effectively  address  the  student’s 
needs  through  coaching,  remedial  help,  and  appropriate  follow  up  exercises. 

•  CVW  software  adds  features  lacking  from  commercial  and  academic  products  such  as  a  room  paradigm  and  object- 
oriented  characters  to  occupy  a  room  and  provides  its  customers  a  broad  array  of  collaborative  technology  to  best  fit 
their  needs. 

Contact  Information:  Bradley  Goodman  617-271-2085  “bsoodman@mitre.org” _ 

FT.  KNOX  ARMOR  CENTER  &  SCHOOL 

Example-Courseware  Development 

•  Sample  -  Fort  Knox  Armor  Center  and  School 

•  Purpose — Task  management  process  and  development  of  automation  supported  course  material.  Armor  Center 
multimedia  distance  learning  initiatives 

•  Infrastructure — CAN  &  link  with  Kentucky  Teletraining  Learning  Network 

•  Courseware  development  — Task  Database  and  Standard  Army  Training  System  (SATS)  Management 

•  Observations 

•  Army  wide  system  for  automated  training  &  doctrine  development 

•  Task  reviews  by  Proponent  Schools  and  utilization  of  Task  Performance  Support  Codes 

•  Shared  dual  use,  internet  utilization  and  access. 

Contact  Information:  Mike  Kelley  502-624-3346  “kelle\m@ftknox-dtdd-emh5. army. mil” _ 
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COMBINED  ARMS  CENTER  &  FT  LEAVENWORTH 

Example-Networked  Students  &  Instructors 

•  Sample  -  Combined  Arms  Center  &  Fort  Leavenworth 

•  Purpose — Officer/Leadership  Training  and  Education  of  over  1 ,500  resident  students  (CGSOC,  CAS3,  PCC,  & 
WARFIGHTER  Exercises)  and  8,000  plus  non-resident  students.  WARRIOR  (Institution)  and  WARFIGHTER  (Unit) 
applications  of  Army  Training  XXI. 

•  Infrastructure— WAN,  CAN,  CACNET  and  Internet  Networks 

•  Courseware  development  — TATS-C  and  DL  initial  stages  of  courseware  redesign 

•  Observations 

•  Army  wide  Research  Library  being  established  through  Center  for  Army  Lessons  Learned  (CALL) 

•  Aggregate  Level  Simulations  Protocol  (ALSP)  established  in  SIMNET  possible  application  for  Simulations  experiences 
in  DL 

•  CAC  Ft.  Leavenworth  CALL,  CGSC  and  National  Simulations  (NSC)  Technology  Innovation  opportunities  offer  DL 
applications  in  educational  development. 

•  TNET,  VTT  and  PicTel  video  training  connectivity 

Contact  Information :  LTV  Rolf 913-684-4713  “rolf@leav-emh Larmy.mil” _ _ _ 

STATE  OF  GEORGIA 

Example-Networked  Classrooms 

•  Sample  -  State  of  Georgia 

•  Purpose — State-wide  government  &  education  network  links  for  interactive  training  and  distance  learning  classroom 
applications. 

•  Infrastructure — State  connection  to  375  sites  via  T1  line;  another  41  Telemedicine  sites  within  the  state,  and  a  Satellite 
down-link  to  over  2,200  sites. 

•  Observations 

•  Leverage  Statewide  terrestrial  and  satellite  infrastructure  to  provide  DL  classrooms  for  Reserve  Component  and  possibly 
Active  Army  usage 

•  Consortium  and  Partnership  opportunities  exist 

See  Georgia  State  University  entry  for  related  information 

Contact  Information^  Bob  Reese  “breese @doas. state. sa.us” _ _ _ 

INSTITUTE  OF  ELECTRICAL  &  ELECTRONIC  ENGINEERS 

Example-Standards  Development 

•  Sample  -  Institute  for  Electrical  &  Electronic  Engineers  (IEEE) 

•  Purpose — working  &  study  groups  to  develop  standards,  guidelines,  and  recommended  practices  for  computer-based 
learning. 

•  Computer  Society  Standards  Activities  Board  (SAB);  IEEE;  Industry  &  Academic  Professionals  collaborate  in  working 
&  study  groups. 

•  Standards  development — Reference  Model/Architecture;  Learner  Model;  Dictionary;  Task  Model;  Learning 
Agreement;  Session  Management  Language;  Protocols  for  Tools/Agents;  Task  Ontology;  and  Authoring  Tools. 

•  Observations 

•  Courseware  modularity  requires  definition  of  technical  standards  for  developing  content  in  the  basic  modules  first. 

•  Standards  will  provide  enabling  tools,  courseware,  information,  and  services  on  a  component  basis. 

•  Through  a  distributed  approach;  encouraging  the  development  of  many  different  standards,  with  networked  SMEs, 
organizations  and  consortiums,  a  natural  selection  of  standards  will  form  comerstone  building  blocks  to  eventually 
deliver  workable  learning  systems  based  on  successful  standards. 

•  The  field  of  computer-based  learning  (or  Interactive  Multimedia  Instruction)  has  a  need  for  a  wide  variety  of  technical 
standards.  The  TF.F.F.  P1484  Working  and  Study  Groups  (10  groups  to  date,  and  growing)  are  working  to  identify  and 

_ develop  such  standards.  Representatives  of  the  U.S.  Army  lead  the  creation  of  this  initiative  and  remain  active.  With 
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the  Army’s  Education  and  Training  Challenge,  these  efforts  of  the  IEEE  and  the  active  role  these  representatives  are 
taking  in  the  development  of  the  needed  standards  should  be  accelerated. 


Contact  Information:  James  Schoenins  908-532-0118  “schoenin@doim6.monmouth.arm\.mil” 


UNION  CITY 

Example-Networked  Students  &  Instructors 

•  Sample  -  Union  City,  New  Jersey 

•  Observations:  The  Union  City  project  has  to  be  considered  a  major  success.  It  has  improved  the  scores  of  children  in  urban 
inner-city  environment.  In  addition,  it  has  served  as  a  catalyst  to  focus  the  cooperation  and  attention  of  major  players  in  the 
educational  community.  The  community  as  a  whole  has  rallied  around  this  project,  and  the  results  of  five  to  six  years  of  effort 
are  amazing!  Who  has  been  involved  and  made  major  contributions?  The  state  of  New  Jersey  changed  the  funding  profile  for 
public  school  education.  The  Union  City  Board  of  Education  got  the  teachers’  union  to  give  up  certain  programs  in  order  to 
fund  this  new  initiative.  Teachers  re-designed  the  curricula.  Bell  Atlantic’s  foundation  provided  scholarships  to  teachers  who 
wanted  to  utilize  new  technology.  Bell  Atlantic’s  foundation  provided  scholarships  to  teachers  who  wanted  to  utilize  new 
technology.  Bell  Atlantic  provided  access  and  computers  in  the  homes  of  150  students.  The  entire  community  exhibited  the 
patience  to  start  this  program  and  to  wait  until  it  had  advanced  enough  to  start  classes.  Finally  the  parents  let  their  children 
have  a  new  experience.  The  project  started  in  1987  that’s  a  decade  ago.  It  took  3  years  to  get  the  courses  redesigned  and  the 
teachers  ready  to  deliver  then  in  this  new  environment.  It  took  another  3  years  to  get  the  technology  into  place.  Since  then  the 
bandwidth  required  to  support  the  enterprise  decreased  from  T1  access  to  ISDN  access.  Two  years  later  truancy  had  almost 
disappeared,  test  scores  had  been  raised  from  60%  failure  rates  on  state  wide  normal  tests  to  the  point  where  the  children  score 
10%  higher  than  average  in  all  subjects.  The  last  measure  of  success  is  buy-in.  Today  more  children  apply  for  admission  than 
there  are  tenant  spaces  in  Union  City!  The  project  continues  to  evolve.  Visit  their  home  page  on  the  world  wide  web. 

Contact  information :  http://www.union-citv.kl2.nhus  _ 


WORLD  LECTURE  HALL 

Example-Internet  DL  Courses 

•  Sample  -  World  Lecture  Hall 

•  Observations:  The  World  Lecture  Hall  is  a  compendium  of  courses  that  are  available  over  the  world  wide  web.  The  subject 
matter  ranges  from  accounting  to  virology.  Classes  for  over  ninety  general  subjects  are  cataloged.  Each  category  contains  a 
number  of  sub-categories.  An  example  of  selecting  Atmospheric  Sciences  yields  a  course  on  “Our  Changing  Atmosphere” 
whose  home  page  Illustrates  the  following: 


OUR  CHANGING  ATMOSPHERE 


Syllabus  WeatherNet 

Grading  Weekly  Work 

Schedule  Lecture  Notes 


News/Updates  Query 

Contact  Information:  http://www.  utexas.  edu/world/lecturef 
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APPENDIX  E 

ACADEMIC  INSTITUTIONS 
OFFERING  GRADUATE  DEGREES 

VIA 

DISTANCE  LEARNING 
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Universities  With  Non-Graduate  Distance  Learning  Programs: 


Adams  State  College 
America  Academy  of  Nutrition 
American  College  of  Prehospital  Medicine 
American  Open  U. 

Arizona  State  University 
Athabasca  University 
Atlantic  Union  College 
Bemidji  State  University 
Board  of  Governors  Universities 
Boston  University 
Boyce  Bible  School 
Brigham  Young  University 
Burlington  College 
Caldwell  College 
Carnegie  Mellon  U. 

Charter  Oaks  State  College 
Chemeketa  Community  College 
Clemson  University 
Cleveland  Institute  of  Electronics 
Columbia  Union  College 
Colombia  University 
Dalhousie  University 
The  Defiance  College 
Diversity  University 
Duke  University 
East  Tennessee  State  University 
Eastern  Kentucky  U. 

Eastern  Oregon  State  College 
Eckard  College 
Elizabethtown  College 
Grantham  College  of  Engineering 
Griggs  University 

Global  Electronic  Multimedia  University 

Global  Laboratory 

Global  School  Net 

ICS  Center  for  Degree  Studies 

Illinois  State  University 

Indiana  Institute  of  Technology 

Indiana  University 

Institute  for  Global  Learning 

Johnson  State  College 

Judson  College 

Kansas  Newman  College 

Kansas  State  University 

Kent  State  University 

Lee  College 

Mary  Baldwin  College 

Marywood  College 

Mid- America  Bible  College 

MIT 

Moody  Bible  Institute 
Mott  Community  College 
Murray  State  University 

National  Institute  for  Paralegal  Arts  and  Sciences 


New  York  Institute  of  Technology/Online  Campus 

North  Central  Bible  College 

Northwood  University 

Ohio  University 

Oklahoma  City  University 

Open  Learning  Agency 

Open  Learning  Fire  Service  Program 

Oral  Roberts  University 

Oregon  State  University 

The  Paralegal  Institute,  Inc. 

Pennsylvania  State  University 
People's  College 
Pikes  Peak  Community  College 
Prescott  College 

Regents  College,  The  University  of  The  State  of 
New  York 

Roger  Williams  University 
Southeastern  College  of  the  Assemblies  of  God 
Southern  Illinois  University 
Southwest  Texas  State  University 
Southwestern  Adventist  College 
Southwestern  Assemblies  of  God  University 
State  College  and  University  Systems  of  West 
Virginia 

State  University  System  of  Florida  External  Degree 
Program 

Stephens  College  School  of  Continuing  Education 
Trinity  College-Connecticut 
Troy  State  University  Montgomery 
University  of  California  at  Berkeley 
University  of  Connecticut 
University  of  Georgia 
University  of  Hawaii 
University  of  Iowa 
University  of  Kansas 
University  of  Michigan 
University  of  Mississippi 
University  of  Missouri 
University  of  Maryland  University  College 
University  of  Nevada 
University  of  Pittsburgh 
University  of  Southern  Colorado 
University  of  Wisconsin-Platteville,  River  Falls, 
Superior 

Upper  Iowa  University 
Vincennes  University 
Virtual  On-line  University 
Washington  State  University 
Weber  State  University 
World  College 

Other  Possibilities: 

University  of  North  Texas 
University  of  Northern  Iowa 
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University  of  Southern  Mississippi 
University  of  South  Dakota 
University  of  Texas  at  Austin 
University  of  the  State  of  New  York  / 

Regents  College 
University  of  Utah 
University  of  Washington 
University  of  Waterloo 
Usenet  University  -  Global  Network  Academy 
Utah  State  University 
Vanderbilt  University 
Vassar  College 
Western  Michigan  University 
Yale  University 


APPENDIX  F 

THEORETICAL  BASIS  OF  LEARNING 

(BEHAVIORISM,  COGNITIVE 
PSYCHOLOGY  &  CONSTRUCTIVISM) 
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The  Theoretical  Basis  of  Learning 


Behaviorism  (1950s- 1960s) 

—  Central  Metaphor:  People  are  animals 

—  Source  of  Knowledge:  codified  sources 

—  Instructor  Role:  Dispenser  of  external  rewards  and  punishments 

—  Goals:  Knowledge  &  Procedures 

—  Instructional  Features:  Immediate  feedback,  no  errors  in  learning  via 

small  increments/steps  using  drill  and  practice 

—  Assessment:  multiple  choice  testing 


•  Cognitive  Psychology  (1960s-Present) 

—  Central  Metaphor:  People  are  computers  (e.g.,  memory  is  long  term 

(hard  drive)  and  short  term  (RAM) 

—  Source  of  Knowledge:  experts  &  codified  knowledge 

—  Instructor  Role:  Dispenser  of  information.  Training  is  facilitating  the 

integration  of  new  information  with  student’s  existing  knowledge 

—  Goals:  Knowledge,  Procedures,  and  Problem  Solving 

—  Instructional  Features:  Delayed  feedback,  errors  used  as  learning 

opportunity  (synchronous/asynchronous)  with  lecture/discussion 

—  Assessment:  Performance  Assessment  (e.g.,  simulation) 


Constructivism  (1980s-Present) 

—  Central  Metaphor:  People  are  knowledge  constructors  in  social 

contexts 

—  Source  of  Knowledge:  codified  distributed  knowledge  &  search 

—  Instructor  Role:  A  guide  for  exploring  individual  and  collective  tasks 

—  Goals:  Problem  Solving,  Deep  Understanding,  and  Self-Regulation 

—  Instructional  Features:  Discussion,  guided  discovery,  collaborative 

learning,  extensive  use  of  context 

—  Assessment:  Performance  Assessment  and  Qualitative  (ethnographic) 

approach 
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APPENDIX  G 


COMPARISON  TABLE  SHOWING 
COURSEWARE  CONVERSION 
COSTS  UNDER  EXISTING  ADLP  AND 
TWO  ACCELERATED  COURSE 
CONVERSION  SCHEDULES 


TATS-C  Conversion  Table 


This  chart  is  based  on  data  in  the  existing  Army  Distance  Learning  Plan.  The  three  columns  under 
Existing  ADLP  provide  numbers  on  courseware  conversion  found  in  the  ADLP  Technical  Volume.  The 
columns  under  30%  Conversion  and  50%  Conversion  are  based  on  accelerated  courseware  development 
schedules  and  represent  courseware  costs  with  increased  initial  conversion  funding  and  immediate 
reinvestment  ofPer-Diem  savings  back  into  the  conversion  process.  This  illustrates  how  accelerated 
courseware  development  can  result  in  a  rapid  return  on  investment. 


Existing  ADLP 


30%  Conversion 


50%  Conversion 


Cost  for 
Conversion 


Courses  Cost J 

Converted  Course 


PD  Cost  Courses  PD  Cost  Courses 

Avoided  Converted  Avoided  Converted 


98 

19,737 

99 

21,396 

00 

13,223 

01 

17,745 

02 

20,036 

03 

25,840 

04 

28,764 

05 

24,369 

06 

26,664 

07 

20,812 

08 

35,760 

09 

9,995 

10 

4,609 

Total 

268,950 

APPENDIX  H 

CARNEGIE  MELLON  UNIVERSITY 
PRESENTATION 
ON  THE 

EFFECTIVENESS  OF  INTELLIGENT 
COMPUTER  AIDED  INSTRUCTION 
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Appendix  H 

Effectiveness  of  Intelligent 
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(few  studies,  some  negative) 
Cognitive  tutors  =  1  sd 


ACT:  A  Cognitive  Theory  of 
Learning  and  Performance 
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Anderson,  J.R.  (1993).  Rules  of  the  Mind.  Lawrence  Erlbaum. 


Cognitive  Tutors:  A  Technology  to 
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problem  ->  context-sensitive  instruction 

Knowledge  Tracing:  Assesses  student's  knowledge  growth  - 
individualized  activity  selection  and  pacing 


Cognitive  Tutor  Research: 
Demonstrations  of  Effectiveness 
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Anderson,  J.  R.,  Corbett,  A.  T.,  Koedinger,  K.  R.,  &  Pelletier,  R.  (1995).  Cognitive 
tutors:  Lessons  learned.  The  Journal  of  the  Learning  Sciences ,  4  (2)  167-207 


You  can  prove  certain  things  from  the 
diagram.  What  might  help  you  prove 
AABC  =  AADE? 
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Client-centered  Engineering 


Cognitive  Tutors 

Practical  Equation  Practical  Pascal 

Algebra  Solving  Geometry  Programming 

Tutor  Tutor  Tutor  Tutor 
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at  these  plans  over  a  range  of  airtime  from  0  to  500  minutes  per  month,  and  how  much  airtime 
that  vou  can  get  per  month  with  each  of  these  plans  for  a  total  cost  of  $100.  She  also  makes  it 
very  clear  that  you  must  include  the  range  of  airtime  for  which  each  plan  is  the  cheapest. 
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U.S.  Army  Combined  Arms  Center/Commandant,  Command  and  General  Staff  College,  Ft.  Leavenworth,  KS 
66027-5000  1 

Deputy  Commanding  General,  U.S.  Army  Training  and  Doctrine  Command  for  Combined  Arms  Support/ 

Commander,  U.S.  Army  Combined  Arms  Support  Command  and  Ft.  Lee,  Ft.  Lee,  VA  23801  -6000  1 

Commander,  U.S.  Army  Aviation  Center  and  Ft.  Rucker/Commandant,  U.S.  Army  Aviation  School/Commandant, 

U.S.  Army  Aviation  Logistics  School  (Ft.  Eustis),  Ft.  Rucker,  AL  36362-5000  1 

Commander,  U.S.  Army  Signal  Center  and  Ft.  Gordon/Commandant,  U.S.  Army  Signal  School,  Ft.  Gordon,  GA 

30905-5000  1 

Commandant,  U.S.  Army  War  College,  Carlisle  Barracks,  PA  17013-5050  1 

Commander,  U.S.  Army  Air  Defense  Artillery  Center  and  Ft.  Bliss/Commandant,  U.S.  Army  Air  Defense  Artillery 

School,  Ft.  Bliss,  TX  79916-5000  1 

Commander,  U.S.  Army  John  F.  Kennedy  Special  Warfare  Center  and  School,  Ft.  Bragg,  NC  28307-5000  1 

Commander,  U.S.  Army  Engineer  Center  and  Ft.  Leonard  Wood/Commandant,  U.S.  Army  Engineer  School, 

Ft.  Leonard  Wood,  MO  65473-5000  1 

Commander,  U.S.  Army  Quartermaster  Center  and  School/Deputy  Commander,  U.S.  Army  Combined  Arms 

Support  Command  and  Ft.  Lee/Commandant,  U.S.  Army  Quartermaster  School,  Ft.  Lee,  VA  23801-6000  1 

Commander,  U.S.  Army  Infantry  Center  and  Ft.  Benning/Commandant,  U.S.  Army  Infantry  School,  Ft.  Benning, 

GA  31905-5000  1 

Commander,  U.S.  Army  Chemical  and  Military  Police  Centers  and  Ft.  McClellan/Commandant,  U.S.  Army 

Military  Police  School,  Ft.  McClellan,  AL  36205-5000  1 

Commander,  U.S.  Army  Ordnance  Center/Commandant,  U.S.  Army  Ordnance  School,  Aberdeen  Proving  Ground, 

MD  21005-5201  1 

Commander,  U.S.  Army  Field  Artillery  Center  and  Ft.  Sill/Commandant,  U.S.  Army  Field  Artillery  School, 

Ft.  Sill,  OK  73503-5000  1 

Commander,  U.S.  Army  Transportation  Center  and  Ft.  Eustis/Commandant,  U.S.  Army  Transportation  School, 

Ft.  Eustis,  VA  23604-5000  1 

Commander,  U.S.  Army  Armor  Center  and  Ft.  Knox/Commandant,  U.S.  Army  Armor  School,  Ft.  Knox,  KY 

40121-5000  1 
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Commander,  U.S.  Army  Intelligence  Center  and  Ft.  Huachuca/Commandant,  U.S.  Army  Intelligence  School, 

Ft.  Huachuca,  AZ  85613-6000  1 

Commandant,  U.S.  Army  Ordnance  Missile  and  Munitions  Center  and  School,  Redstone  Arsenal,  AL  35897-6000  1 

Commandant,  Army  Logistics  Management  College,  Ft.  Lee,  VA  23801-6053  1 

Director,  U.S.  Army  Training  and  Doctrine  Command  Analysis  Center,  Ft.  Leavenworth,  KS  66027-5200  1 

Commander,  Battle  Command  Battle  Lab,  ATTN:  ATZL-CDB,  415  Sherman  Ave.,  Ft.  Leavenworth,  KS 

66027-5300  _  1 

Commander,  Battle  Command  Battle  Lab,  ATTN:  ATZH-BL,  Ft.  Gordon,  GA  30905-5299  1 

Commander,  Battle  Command  Battle  Lab,  ATTN:  ATZS-BL,  Ft.  Huachuca,  AZ  85613-6000  1 

Commander,  Combat  Service  Support  Battle  Lab,  ATTN:  ATCL-B,  Ft.  Lee,  VA  23801-6000  1 

Commandant,  Depth  and  Simultaneous  Attack  Battle  Lab,  ATTN:  ATSF-CBL,  Ft.  Sill,  OK  73503-5600  1 

Commandant,  Dismounted  Battle  Space  Battle  Lab,  ATTN:  ATSH-WC,  Ft.  Benning,  GA  31905-5007  ^  1 

Commander,  Early  Entry  Lethality  and  Survivability  Battle  Lab,  ATTN:  ATCD-L,  Ft.  Monroe,  VA  23651-5000  1 

Commander,  Mounted  Battle  Space  Battle  Lab,  ATTN:  ATZK-MW,  Ft.  Knox,  KY  40121-5000  1 

Commander,  Battle  Lab  Integration,  Technology  and  Concepts  Directorate,  ATTN:  ATCD-B,  Ft.  Monroe,  VA 

23651-5000  1 

Program  Executive  Officer,  Armored  Systems  Modernization,  ATTN:  SFAE-ASM,  Warren,  Ml  48397-5000  1 

Program  Executive  Officer,  Aviation,  ATTN:  SFAE-AV,  4300  Goodfellow  Blvd.,  St.  Louis,  MO  63120-1798  1 

Program  Executive  Officer,  Command,  Control  and  Communications  Systems,  ATTN:  SFAE-C3S,  Ft.  Monmouth, 

NJ  07703-5000  1 

Program  Executive  Officer,  Field  Artillery  Systems,  ATTN:  SFAE-FAS,  Picatinny  Arsenal,  NJ  07806-5000  1 

Program  Executive  Officer,  Intelligence  and  Electronic  Warfare,  ATTN:  SFAE-IEW,  Ft.  Monmouth,  NJ 

07703-5000  1 

Program  Executive  Officer,  Missile  Defense,  ATTN:  SFAE-MD,  P.O.  Box  16686,  Arlington,  VA  22215-1686  1 

Program  Executive  Officer,  Standard  Army  Management  Information  Systems,  ATTN:  SFAE-PS,  9350  Hall  Rd., 

Suite  142,  Ft.  Belvoir,  VA  22060-5526  1 

Program  Executive  Officer,  Tactical  Missiles,  ATTN:  SFAE-MSL,  Redstone  Arsenal,  AL  35898-8000  1 

Program  Executive  Officer,  Tactical  Wheeled  Vehicles,  ATTN:  SFAE-TWV,  Warren,  Ml  48397-5000  1 

Program  Executive  Officer,  Cruise  Missiles  Project  and  Unmanned  Aerial  Vehicles  Joint  Project,  ATTN:  PEO-CU, 

Washington,  DC  20361-1014  1 

Program  Executive  Officer,  Combat  Support  Systems,  ATTN:  AF  PEO  CB,  1090  Air  Force  Pentagon,  Washington, 

DC  20330-1090  1 

Program  Executive  Officer,  Joint  Program  Office  for  Biological  Defense,  5201  Leesburg  Pike,  Suite  1200, 

Skyline  #3,  Falls  Church,  VA  22041  -3203  1 

Program  Manager,  Comanche  Program  Office,  ATTN:  SFAE-AV-RAH,  4300  Goodfellow  Blvd.,  St.  Louis,  MO 

63120-1798  1 

Program  Manager  for  Chemical  DeMilitarization,  ATTN:  SFAE-CD-Z,  Aberdeen  Proving  Ground,  MD 

21010-5401  1 

Superintendent,  U.S.  Army  Military  Academy,  West  Point,  NY  10996  1 

NAVY 

Secretary  of  the  Navy,  Pentagon,  Room  4E686,  Washington,  DC  20350  1 

Under  Secretary  of  the  Navy,  Pentagon,  Room  4E714,  Washington,  DC  20350  1 

Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition),  Pentagon,  Room  4E732,  Washington, 

DC  20350  1 

Chief  of  Naval  Operations,  Pentagon,  Room  4E674,  Washington,  DC  20350  1 

Vice  Chief  of  Naval  Operations,  Pentagon,  Room  4E636,  Washington,  DC  20350  1 

Commandant,  U.S.  Marine  Corps,  Pentagon,  Room  4E714,  Washington,  DC  20380  1 

Naval  Research  Advisory  Committee,  800  N.  Quincy  Street,  Arlington,  VA  22217-5660  1 

President,  Naval  War  College,  Code  00,  686  Cushing  Rd.,  Newport,  Rl  02841-1207  1 
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AIR  FORCE 

Secretary  of  the  Air  Force,  Pentagon,  Room  4E871 ,  Washington,  DC  20330  1 

Under  Secretary  of  the  Air  Force,  Pentagon,  Room  4E886,  Washington,  DC  20330  1 

Assistant  Secretary  of  the  Air  Force  (Acquisition),  ATTN:  SAF/AQ,  Pentagon,  Room  4E964,  Washington,  DC 

20330  1 

Chief  of  Staff,  United  States  Air  Force,  Pentagon,  Room  4E924,  Washington,  DC  20330  1 

Vice  Chief  of  Staff,  United  States  Air  Force,  Pentagon,  Room  4E936,  Washington,  DC  20330  1 

Air  Force  Scientific  Advisory  Board,  Pentagon,  Room  5D982,  Washington,  DC  20330  1 

President,  Air  War  College,  325  Chennault  Circle,  Maxwell  Air  Force  Base,  AL  36112-6427  1 

OSD 

Secretary  of  Defense,  Pentagon,  Room  3E880,  Washington,  DC  20301  1 

Deputy  Secretary  of  Defense,  Pentagon,  Room  3E944,  Washington,  DC  20301  1 

Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Pentagon,  Room  3E933,  Washington,  DC  20301  1 

Under  Secretary  of  Defense  (Personnel  and  Readiness),  Pentagon,  Room  3E764,  Washington,  DC  20301  1 

Under  Secretary  of  Defense  for  Policy,  Pentagon,  Room  4E808,  Washington,  DC  20301  1 

Under  Secretary  of  Defense  (Comptroller/Chief  Financial  Officer),  Pentagon,  Room  3E822,  Washington,  DC  20301  1 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications  and  Intelligence),  Pentagon,  Room  3E172, 

Washington,  DC  20301  1 

Assistant  Secretary  of  Defense  for  Economic  Security,  Pentagon,  Room  3E808,  Washington,  DC  20301  1 

Deputy  Under  Secretary  of  Defense  for  Advanced  Technology,  Pentagon,  Room  3E1045,  Washington,  DC  20301  1 

Deputy  Under  Secretary  of  Defense  for  Acquisition  Reform,  Pentagon,  Room  3E1034,  Washington,  DC  20301  1 

Deputy  Under  Secretary  of  Defense  for  Environmental  Security,  Pentagon,  Room  3E792,  Washington,  DC  20301  1 

Principal  Deputy  Under  Secretary  of  Defense  for  Acquisition  and  Technology,  Pentagon,  Room  3E1006, 

Washington,  DC  20301  1 

Chairman,  Joint  Chiefs  of  Staff,  Pentagon,  Room  2E872,  Washington,  DC  20318-9999  1 

Vice  Chairman,  Joint  Chiefs  of  Staff,  Pentagon,  Room  2E860,  Washington,  DC  20318-9999  1 

Director,  Operational  Test  and  Evaluation,  Pentagon,  Room  3E318,  Washington,  DC  20301-1700  1 

Director,  Defense  Research  and  Engineering,  Pentagon,  Room  3E1014,  Washington,  DC  20301-3030  1 

Director,  Defense  Advanced  Research  Projects  Agency,  3701  N.  Fairfax  Dr.,  Arlington,  VA  22203-1714  1 

Director,  Ballistic  Missile  Defense  Organization,  Pentagon,  Room  1  El  081,  Washington,  DC  20301-7100  1 

Director,  Defense  Airborne  Reconnaissance  Agency,  Pentagon,  Room  4C100,  Washington,  DC  20301-3160  1 

Director,  Defense  Information  Systems  Agency,  701  S.  Courthouse  Rd.,  Arlington,  VA  22204-2199  1 

Director,  Defense  Intelligence  Agency,  Pentagon,  Room  3E258,  Washington,  DC  20301-7400  1 

Director,  Defense  Intelligence  Agency  Missile  and  Space  Intelligence  Center,  Building  4505,  Redstone  Arsenal,  AL 

35898-5500  1 

Director,  Defense  Logistics  Agency,  8725  John  J.  Kingman  Rd.,  Suite  2533,  Ft.  Belvoir,  VA  22060-6221  1 

Director,  Defense  Mapping  Agency,  8613  Lee  Highway,  M.S.  A-1 ,  Fairfax,  VA  22031-2137  1 

Director,  Defense  Nuclear  Agency,  6801  Telegraph  Rd.,  Alexandria,  VA  22310-3398  1 

Director,  Defense  Security  Assistance  Agency,  1111  Jefferson  Davis  Highway,  Suite  303,  Arlington,  VA  22202  1 

Director,  National  Security  Agency,  9800  Savage  Rd.,  Ft.  Meade,  MD  20755  1 

Director,  On-Site  Inspection  Agency,  201  W.  Service  Rd.,  Dulles  International  Airport,  P.O.  Box  17498, 

Washington,  DC  20041-0498  1 

Defense  Science  Board,  Pentagon,  Room  3D865,  Washington,  DC  20301  .  1 

Commandant,  Defense  Systems  Management  College,  9820  Belvoir  Rd.,  Suite  G-38,  Ft.  Belvoir,  VA  22060-5565  1 

President,  National  Defense  University,  300  5th  Avenue,  Ft.  McNair,  Washington,  DC  20319-5066  1 

Commandant,  Armed  Forces  Staff  College,  7800  Hampton  Blvd.,  Norfolk,  VA  23511-1702  1 

Commandant,  Industrial  College  of  the  Armed  Forces,  408  4th  Ave.,  Bldg.  59,  Ft.  McNair,  Washington,  DC 

20319-5062  '  1 

Commandant,  National  War  College,  Washington,  DC  20319-5066  1 
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OTHER 

Defense  Technical  Information  Center,  ATTN:  DTIC-OCP,  8725  John  J.  Kingman  Rd.,  Suite  0944,  Ft.  Belvoir, 

VA  22060-6218  1 

Director,  Central  Intelligence  Agency,  Washington,  DC  20505  1 

National  Research  Council,  Division  of  Military  Science  and  Technology,  Harris  Bldg  Rm.  258, 

2101  Constitution  Avenue  NW,  Washington  DC  20418  1 

Director,  Institute  for  Defense  Analyses,  ATTN:  TISO,  1801  N.  Beauregard  St.,  Alexandria,  VA  2231 1-1772  1 

Library  of  Congress,  Exchange  and  Gift  Division,  Federal  Document  Section,  Federal  Advisory  Committee  Desk, 

Washington,  DC  20540  1 
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